This third post in the “Who do SharePoint Projects Fail” series has been hard to write, not because I am struggling with ideas, but because I have too many! It is hard to put all the reasons why SharePoint projects go wrong into a coherent chain of logic.

In the first two posts in this series, we did a basic examination of wicked problem theory.

Part 1 introduced you to tequila slammers, as well as the pioneering work by Horst Rittel and the concept of wicked problems.

Part 2 also delved into the murky depths of academic history to demonstrate that even back in the seventies when ABBA stole the hearts and minds of teenyboppers around the world, at least some people had time to look at wicked problems in relation to building IT systems.

If you take away anything from part 1 and 2, it is this.

• Too many tequila slammers hurt
• Before you blame the product, the project manager, the stakeholders, the nerds, the methodology or anything else in vicinity, go back to the problem you are solving and determine its ‘wickedness’

Now we will finally look at this large, complex, scary beast known as SharePoint. I have no means to quantify how much of a percentage of project problems arise from issues related to “the product”, but it definitely happens. Unsurprisingly enough, it is easy to argue that some of the areas that I highlight below are people issues, but we still get to indulge in Microsoft bashing – and who doesn’t enjoy a bit of that eh?

The Microsoft Effect – Good enough…

Microsoft in some ways remind me of Brittany Spears*. Both crash things, both can be abusive, both get themselves into trouble with the law, both have ignored legal judgements, both have loyal fans who love them no matter what and detractors that will never accept they can do anything right, both get caught with their pants down on occasion, and both have produced some pretty ordinary crap in their time. Despite this, they sell squillions of copies and make a ton of money.

*Just in case you read this article at some future point where Brittany is long forgotten, substitute whichever starlet is in rehab

But Microsoft is *big*, Microsoft are *safe*. We already use Microsoft in the organisation now and thus Microsoft products *integrate well*.

So what is it about SharePoint that makes it such a magnet for a wicked project? Put quite simply, with SharePoint they have taken integration to another level. Microsoft have done their homework and created a product/platform that has a combination of the right kind of capabilities that *together* can be extremely compelling. Each component, judged individually as a point solution would not necessarily compete against dedicated products in the space. But Microsoft long ago learned that a combination of tight integration and the old 80/20 rule usually wins.

There have been document management systems and web content management for years. I’ve previously been involved with a large Hummingbird deployment. As soon as I saw MOSS 2007 I know it was going to massive, not because it was as good as Hummingbird, because it wasn’t. But it was *good enough* and it was made by Microsoft. Market success therefore was assured.

Easy to Impress – A Mythical Example

To demonstrate how effective this combination of features can be as a sales tool, consider this mythical conversation between a somewhat green, yet enthusiastic Microsoft evangelist named Bill Gates, who is discussing MOSS with key stakeholders at the company GOOSUNACLE. Their names are Scott McNealy, Sergei Brin, Steve Jobs and Larry Ellison.

(1 point for each in-joke you spot)

Scott McNealy (Infrastructure Manager): Okay Bill, here’s our key problem. We can’t find anything and our knowledge sharing is abysmal. We waste so much time trying to work out where stuff should be, and nobody follows the standards. People reinvent the wheel constantly and we keep making the same mistakes over and over again. Our file system really sucks with so much duplication and complexity. It has so much history there, that nobody wants to own it or fix it up. This has cost us market share and our bottom line is suffering. What can you do to help?

Bill Gates: Well Scotty, (do you mind if I call you Scotty?), I am glad you asked. SharePoint represents a new way to manage your enterprise documents. It can help you with built in version control, check-in/out, metadata classification and a recycle bin. Additionally, it is web based, making it much more accessible to your staff. It is also tightly integrated with Microsoft Office, particularly the 2007 version. We also have document format conversion features and records management too.

Sergei Brin (Knowledge Manager) : “Sweet!. Kind of makes the whole notion of a separate intranet and file system start to look ‘old school’. What about search? I feel very strongly that a good search engine is worth its weight in gold.

Bill Gates: That’s a great point you make Sergei, but I think it is best you leave search to us experts at Microsoft. Now in relation to your question, search is vitally important and SharePoint has a very powerful, scalable, customisable search engine that allows you to index all sorts of data sources. 

Larry Ellison (Database Administrator). Okay, so you can classify and search your files and other content. But I’m a database kind of guy. What about our line of business data in our databases? Can we improve its visibility to a larger audience?

Bill Gates: You certainly can Larry. We have several features that facilitate this, but specifically, we have the business data catalogue feature, which allows you to connect to 3rd party data sources for search, business intelligence and reporting. That way, you do not need to train your users in your line of business applications. They can simply visit the SharePoint portal, and view their dashboards and key performance indicators, so gain visibility into the business. In addition we integrate with SQL Reporting Services and can easily surface enterprise reports into the portal. Excel services is another feature that falls into this area. Imagine being able to publish your complex Excel sheets to the portal and users can interact with then without requiring excel on their desktops.

Scott McNealy: That is all very interesting indeed, document management, business intelligence and reporting in the one product. The other big issue we have is ensuring that people do things consistently. Larry and Steve in particular, keep releasing marketing slogans without formal approval. The “unbreakable” and “sooper dooper secure”  campaigns kind of backfired on us, and it would have been much better we had a more rigorous review process.

Bill Gates: Ah yes Scotty my lad, that is an excellent example. SharePoint contains a powerful workflow engine under the hood, that allows us to automate business process. So for example, at Microsoft when staff members leave the company, as part of our exit process we automatically create a task for the ever popular “throw the chair sendoff”. Supporting the workflow, is the document management features that I described earlier, as well as business forms via infopath and forms server.

Steve Jobs (Communications and Branding Manager). What about the GUI? It has to look good for people to engage with it. Additionally, I’d like the content to be available via mobile devices too.

Bill Gates: SharePoint has a powerful web content management foundation as well. It is built upon ASP.NET, which uses a templating system known as master pages, page layouts and web parts. But rather than me get into the detail here, I’ll recommend you visit my all time favourite blog called CleverWorkArounds and read the branding series. It should answer all of your questions. In relation to mobile devices, content can be displayed on these devices with no additional conversion. Suffice to say that it takes advantage of some of the workflow and document management features to facilitate content editorial approval, scheduled publication, and the like.

Steve Jobs: We are sold, Sergei, pass me the chequebook.

The downsides of versatility

So, Bill has sold Steve, Larry, Sergei and Scott on the possibilities of SharePoint. All of their respective buttons have been pushed and that can see it solving particular problems in the business. At a high level, they can see the value of re-use and efficiencies gained from using the same platform.

But which problem are we solving? If you asked them individually at this point, they would give you different answers. Its clear that Steve likes the WCM features, while Larry digs Business Intelligence. Sergey loves search and Scott wants to sort out document management.

Already, there is a wicked problem in the making, because it is easy to fall into the trap of “a tool looking for a problem” and nobody yet has the same definition of the problem. (one of Rittel’s key wicked problem warning signs that we covered in part 1)

The panacea effect

With so many features in SharePoint, combined in just the right way, it is the panacea you have been looking for. So you have found your cure, lets use it for everything right? The panacea solution often costs more, or is overkill for an initial problem, so to justify it requires you to sell it’s potential to solve other, bigger problems.

If you have read the first two articles in this series, you will realise pretty quickly that this is very high risk approach and can get out of hand pretty quickly. Just because a tool is versatile and has the potential to solve multiple problems does not mean it will necessarily be the best option to solve your particular problem.

Add the Microsoft effect to the panacea effect and the combination is more lethal! 

IT departments are particularly guilty of going down the panacea road and the reason this happens is also unsurprising when you think about it. IT are conditioned to think in terms of ‘tools and services’. By that I mean that, they exist to support and manage the organisations IT services. All business divisions have a desire to consolidate, streamline and reduce wastage and the IT department is no exception. In terms of an IT Department perspective, consolidating the number of “IT services” is attractive as there is tangible cost in managing many disparate systems.

So on the surface of it, SharePoint brings the IT department the potential tangible advantage of a common platform to support and manage.

I won’t belabour this point any further, because I have written about this issue on a couple of occasions, the best example being my choose your own adventure post and more specifically, its follow up. To the left is the book cover I did for the choose your own adventure. Sometimes pictures speak a thousand words.

It is also not fair to pick on IT either. I’ve gone out to many sites where the ‘SharePoint champion’ is not an IT person at all. But there is a consistent theme there. They want to use it because they have seen the potential for it to solve problems. Seeing the potential to solve a problem is not a problem statement. Therefore, by deploying this tool into the organisation, what problem are you solving and how do you determine if it has been effectively solved?

As a result of the panacea effect, we end up at another great debate of IT projects.

Point solution vs platform solution

I believe that as far as web content management products go, there are better point solutions around than the features SharePoint supply. WordPress for me is a much better blog system than SharePoint, Drupal as a pure WCM offers similar features to SharePoint for a fraction of the cost. Documentum is a better records management system by far and Cognos is going to trump SharePoint in the business intelligence stakes. Closer to home, in my SharePoint for Cisco nerds series, I had a comment that suggested that RANCID was a much better choice than SharePoint for Cisco configuration backups. He is absolutely right too, it is much better. But he is equally wrong too, depending on what the problem is and the point of view of the individual. (to find out why you have to read the whole series and judge for yourself)

This debate is essentially a version of the “Off the Shelf” vs “Custom Designed Software” chestnut. A point solution will meet your core requirements, however it may require you to change your business processes to work around it. Custom designed solutions can integrate nicely with your business process as they are to your exact specification. But the time and cost to develop can be significantly more and requirements can be fluid or prone to creep. (and you know how project managers hate that!)

Ah, but of course, being a point solution, the advantage of those products are diluted as your requirements broaden. You need multiple point solutions, those point solutions need to play nice with each other. The integration and support costs start to increase.

Although I see SharePoint as a platform solution in general, it actually has the best and worst elements of both off the shelf and custom designed software. It can cost you a lot of dollars and it all depends on your problem being solved. For example, it can be used purely as an application development platform, where a custom solution can take advantage of SharePoint features to underpin their own application, saving development time as you do not have to write certain functionality yourself. In this scenario it was always going to be a custom development job anyway, and SharePoint actually saves you money.

However, some problems are best fixed with a point solution and some are not. Where you can get into trouble is investing a lot of time and effort into a platform, because of the adherence to the panacea effect. The goal behind the platform may well be legitimate and well intentioned, but at the end of day, you are potentially talking a high development cost to get it exactly the way you want when a point solution may be a fraction of the cost.

I previously blogged a 6 part series on SharePoint ROI considerations, and although I haven’t blogged about it yet, there are financial modelling techniques available to consider some of the esoteric considerations like ‘opportunity cost’ of point vs platform. I will blog on this topic in the future, but for now, if you take anything away from this section, it is not to underestimate the value of the point solution when looking at your problem being solved.

Conclusion

*pauses for a big sigh*

That was a tough article to write, and the next one is probably going to be tougher, as I will no doubt have an urge to go back and edit this article. But I’ll resist that temptation as otherwise I’d never get these articles out.

But I hope that what you take away from this post is a deeper understanding of some of the reasons why a SharePoint project in particular can get out of hand and become wicked in nature. The quicker you spot them, the quicker you can head off the potential pain and suffering that will follow.

In the next post we will keep digging around SharePoint’s dirty little secrets and then look into some techniques that we can use to try and mitigate some of these issues.

cheers

Paul

                      

No Tags

Hey there!

Sorry it has taken me a while to get back to the Cisco articles. The “choose your own adventure” post took longer than I thought it would and I also was side tracked blogging about annoying programming issues with XML and Javascript.

This is the last of the Cisco fanboy series of articles and really its more a tidying up of loose ends. To call this last article a Cisco fanboy article is a bit of a stretch actually, since we are now moving to a broader level of governance and accountability, and is therefore not really about Cisco, so I’ll start a new series more appropriately titled and continue from where this article leaves off. 

I started the series with the intent of starting with a seemingly innocuous scenario (Cisco TFTP backups), demonstrating how SharePoint can be leveraged as an okay point solution. I then tried to slowly expand the scope to the broader issues of complex infrastructure management management, while sticking to a Cisco/IP network oriented theme in an attempt to get technical thinkers (like Cisco guys) to think beyond nuts and bolts. This also demonstrated how thinking past ‘the point solution’ can being more substantive benefits. 

Here is the quick recap on what we have covered so far:

• Part 1 illustrated how it it possible to use SNMP to tell a Cisco IOS device to dump its configuration to TFTP. We talked about the version control feature of SharePoint and why it makes sense to TFTP your configs to a SharePoint document library. We covered the WEBDAV network provider supplied with XP and Win2003 and finished off with a basic example using the TFTP server TFTPD32.
• Part 2 went into more detail about the issues you can face when using the WEBDAV network provider. It also went into more detail on 3 TFTP server products (WinAgents TFTP, SolarWinds TFTP Server and TFTPD32 and why TFTPD32 ended up being the best choice for this purpose.
• Part 3 then looked at a wonderful open source TFTP server written in C# called TFTPUtil. We modified the source code of this TFTP server to use the SharePoint SDK and upload files to a SharePoint document library.
• Part 4 continued the examination of the TFTPUtil server and another modification of it to use SharePoint web services to upload files. We then summarised the merits of each of the methods.
• Part 5 switched tack and introduced a real world applicable scenario of combining Cisco configuration backups along with diagrams, documentation, asset information and site/support information into a one-stop-shop integrated portal, utilising lists, libraries and workflow to integrate it all neatly.

As I said in my introduction, I intend to get into some more high level strategic stuff in relation to IT operations/infrastructure management in a forthcoming series of articles. So in this last set of articles, I will cover off one major remaining question that I think my technically minded readers would want to see addressed.

“But what if SharePoint dies”?

This is a very common query/issue I get. (Luckily, I have an answer that has satisfied all so far … touch wood  )

Although I have been able to convince even some of the most fervent anti-Micro$oft, anti-portal people the power that SharePoint can offer for knowledge management and collaboration, people quite rightly get a little nervous about the notion of all of your critical information sitting in a single application like SharePoint.

Certainly, SharePoint  is a damn complex suite of applications, meaning that it’s own backup and recovery is more complex than most applications. It requires forward planning, user and IT staff training, lots of governance, a relatively process mature IT department and a heck of a lot of other effort. “and after all of that you still want me to store all of our critical information there?”

I believe absolutely that it is well worth this effort, and a cheap tool released by the guys at Colligo make the whole question somewhat redundant with the release of their cheap (but excellent) “Colligo Reader”.

Colligo Reader is free for personal use and cheap for corporate use. It allows a one way replication of SharePoint portals for offline access. “Yeah but Outlook and Groove can do this”, I hear you say. but this product synchronises libraries, lists, forms, all the while preserving their views. Sweet!

Imagine the possibilities here. Your network administrators could run a continual sync of your Cisco infrastructure portal on their laptops. In the event of a major catastrophic fault where all IT services are unavailable (including SharePoint), you have an the most up to date copy of your documentation, support information and most recent network configuration.

Colligo Demonstrated

So let’s install the product and synchronise the http://tidemo/tftp site that we used in the previous articles to illustrate how useful Colligo is.

Starting Colligo reader for the first time presents you with a basic GUI as shown below.

The first thing to do is to choose a SharePoint site to synchronise. Click the site drop down and choose to download a site. Enter the URL of the SharePoint site to download.

Colligo Reader will now perform an initial sync of the site. Depending on how much content is stored on the site, this may take some time.

One the initial sync has completed, you see the main Colligo Reader screen, except this time we can see the site listed and the documents and lists that have been synchronised.

Now let’s examine some of the lists and document libraries created in part 5, that have been synchronised.

IT Asset Documentation

Note that this looks somewhat like a SharePoint document library. We have our views on the top right, and we can also see the documents and the metadata columns that were created for this library. (“Related Device” and “Document Type”). Below is the same document library, illustrating the “By Document Type” view.

For reference, here is the original SharePoint view to compare.

“Client Sites” List

Below is the synchronised “Client Sites” list. Like the document library example above, all columns and views are fully synchronised.

Caveats

There are only three issues that spring to mind when using Colligo Reader. All three could be addressed by Colligo is they felt so inclined.

The first caveat is that only the latest version of library/list data is replicated. So those 291 versions of myfile.txt I had in my http://tidemo/tftp site (see below ), only the latest one is replicated to Colligo.

I don’t see this as a major issue, as the whole point here is disaster recovery for when SharePoint is not available.

The second caveat is when a site is replicated, sub-sites are not replicated unless you explicitly add them to be downloaded. It would be great if you could tick a box that says to replicate the current site and all sub-sites.

The final caveat is the major one. I really wish that Colligo offered to password protect synchronised sites (on a site by site basis). While it is great to be able to have an up to date, offline copy of your critical infrastructure information, but some will then point to the obvious security implications of laptops containing this goldmine of sensitive information all nicely laid out for unauthorised parties to access.

My technical answer to this is to look at Vista/bitlocker (or 3rd party equivalent) encryption at an operating system level, since Colligo content isn’t necessarily the only sensitive content on a notebook or PC.

My non technical answer is that this is a typical risk vs reward scenario and the applicability of this solution is really dependent on your security posture.

Conclusion

Colligo Reader is an exceptionally useful tool for the offline synchronisation of certain types of SharePoint portals, in particular IT infrastructure and managed services portals make excellent candidates. Our example above shows that we now have VPN details, customer contacts, documentation, configuration backups and support contract details all readily available without requiring SharePoint being available.

So, when step back and look at all we have covered here (particularly the last two articles), we have managed to kick a few significant goals. What makes this really interesting is that we are using tools and techniques that is applicable not just for Cisco gear, but pretty much any other IT infrastructure, whether it is line of business applications, complex hardware or a combination of both.

Whatsmore, while not replacing the need for and value of a Business Impact Analysis and Disaster Recovery Plan, you have still in a significantly improved position if something bad was to occur.

As I said earlier, I’ll be revisiting this again, now more broadly, looking at this topic from the point of view of an IT Operations Manager who has compliance requirements. They are not interested in Cisco per se, but about the whole enchilada!

until then…

regards

Paul

                      

No Tags

Greetings SharePoint fans and Cisco people who are soon to be SharePoint fans! I’m back with part 5 of this series on leveraging features of MOSS 2007 and WSS 3.0 to help manage Cisco infrastructure.

For those of you who hate programming topics and have no interest in TFTP issues, we may well have finally gotten to a topic that you are interested in! For the rest of you, who have read the first four articles, we now shift our focus from getting Cisco configurations into SharePoint, to doing something useful with them once we have done so.

In this post, we will examine the out of the box capabilities of workflow in SharePoint and delve a little deeper into document libraries and content management. So this is more aimed at Cisco people than it is at SharePoint Pros. SharePoint people, much of the topics I cover in this article you may have seen before.

I personally don’t think this post has a high coffee requirement rating in terms of complexity of concepts, compared to the last two anyhow. However be prepared: it is a long read.

CleverWorkArounds Coffee requirement rating:

Here is the quick recap on what we have covered so far:

• Part 1 illustrated how it it possible to use SNMP to tell a Cisco IOS device to dump its configuration to TFTP. We talked about the version control feature of SharePoint and why it makes sense to TFTP your configs to a SharePoint document library. We covered the WEBDAV network provider supplied with XP and Win2003 and finished off with a basic example using the TFTP server TFTPD32.
• Part 2 went into more detail about the issues you can face when using the WEBDAV network provider. It also went into more detail on 3 TFTP server products (WinAgents TFTP, SolarWinds TFTP Server and TFTPD32 and why TFTPD32 ended up being the best choice for this purpose.
• Part 3 then looked at a wonderful open source TFTP server written in C# called TFTPUtil. We modified the source code of this TFTP server to use the SharePoint SDK and upload files to a SharePoint document library.
• Part 4 continued the examination of the TFTPUtil server and another modification of it to use SharePoint web services to upload files.

I think the best way to get stuck into part 5 is via a real-world Cisco scenario. For this scenario, I will (loosely) use one of my roles from a few years back. It was not the biggest or most complex network I’ve managed, but it was quite varied due to the nature of the business they were in.

The scenario

Our scenario company is called CleverWorkArounds Corp and it is a products and services company. This business is involved in digital signage primarily (you know all those LCD screens spamming you with information and advertisements in shopping malls, train stations, cinemas, fancy hotel lobbies, casinos, etc).

Below is an example of large scale digital signage. Look closely…is that a windows error I see?

There are many installations of the product all over the world. Some locations include countries with encryption restrictions or government firewalls. Many of the installations are contracted to be monitored in real-time and in many installs the day to day management is provided by CleverWorkArounds Corp as a managed service.

Including head office in the USA, CleverWorkArounds Corp has 5 offices around the world where the installations are put together and tested before being shipped to site.

The Network

The network infrastructure employed by CleverWorkArounds Corp is Cisco based. The most complex part about it is the VPN connectivity to office and client sites. There is a lot of variation. “Managed Service” type installations typically use CleverWorkArounds Corp supplied network gear on site, in a segmented or isolated network. However many installations are hosted on the client’s infrastructure, often not Cisco, and all CleverWorkArounds Corp has is a VPN connection to the site.

CleverWorkArounds Corp has implemented VLAN’s, wireless and multicast networking at many sites. The security set-up and configuration varies from site to site, as it depends on who owns the infrastructure, security posture of the client and the level of local support on the ground at that site.

Finally, even the VPN technology varies. Most sites are point-to-point IPSEC, but some require CleverWorkArounds Corp to use PPTP or SSL VPN via OpenVPN when firewalls/government restrictions get in the way.

The Issues

The most critical issue with this type of network is security. Not only does CleverWorkArounds Corp have to protect itself from it’s clients, since many client sites are VPN connected with varying levels of security posture, CleverWorkArounds Corp also has to protect client’s from each-other. A relatively complex set of access lists are maintained and authentication to all CleverWorkArounds Corp network devices is via RADIUS and central authentication.

Aside from security, the CleverWorkArounds Corp corporate offices are connected to each-other via GRE tunnels over IPSEC, which allows multicast traffic and as a side effect, dynamic routing protocols. EIGRP is the protocol used. Many ‘lab’ VLAN’s are running at any given time, while new installations are being put together.

On the VPN side, on top of the variation in underlying technology, some Network Address translation is performed where IP address ranges overlap, as many clients like to use the 10.0.0.0 network with a /8 subnet mask (255.0.0.0) and don’t realise how restrictive this can be when doing b2b connectivity.

So What Are We Interested In Here?

From the point of view of the business, all of this has to be up and running, humming along with no service disruption as downtime = lost $. No doubt you have taken out SMARTnet maintenance contracts right? .

So you need to have all the tools and critical information on hand to recover as quickly as possible.

In my career, I’ve been dumped into the deep end a few times, inheriting pathetically maintained/managed infrastructure that is undocumented and a ticking time-bomb. Is this picture familiar?

Thus I always like to leave a site or infrastructure well documented. However my philosophy here is not to rewrite the manual. I do make some assumptions, including the assumption that the next person taking over at least has some semblance of previous experience and understanding of the products making up the infrastructure.

Therefore I do not attempt to rewrite product manuals. I will write a general network SOE document to go with a layer 2 and layer 3 network diagram. The SOE will cover implementation details of the network design including RADIUS, SNMP, EIGRP, Spanning tree, VPN, VLAN’s, IP Telephony, etc. But the SOE will not explain what each of these technologies are and what they do – that is assumed information.

That document is around 40-60 pages usually and once written, it is fairly easy to maintain because it is not filled with constantly changing operation information. That is kept separately. For example I’ll maintain lists of each client site, including the IP range used, details and serial numbers of devices on site, maintenance agreement numbers, client contact details, passwords, VPN details such as the certificate or shared key. I’ll document what each VLAN is for, SNMP community string, etc.

Finally, in case you did not know already , I implement my perl script to back up the configurations of all of the IOS devices to TFTP. (And that perl script is well documented in the network SOE I described earlier).

So this is all very nice and standard practice, and we can do this without SharePoint just fine, but let’s go through an simple example of maintaining this information in SharePoint and see the benefits.

SharePoint Revisited

First up, let’s now modify the SharePoint site (http://tidemo/tftp) we were using for the TFTP experimentation. We will add some additional columns and lists. This will allow us to track much more interesting information than just the backups of router configs.

CleverWorkArounds caveat: This is an example and does not reflect reality! It purely serves to illustrate SharePoint’s capabilities. In addition, this is not meant as a complete newbie SharePoint tutorial or this post would get far too big – I have skipped some details here.

“Clients” list

First up, we have a number of clients and remote sites. So let’s create a contact list to store client and key contact details. From “Site Actions”, choose “Create” and from the list of choices, create a new Contacts list.

We will call this list “Clients” and add some clients in.

“Client Sites” list

Next up, each client can have one or more sites. For each site, we are interested in not only its physical location, but also the IP Address range used at that site, the VPN type and detail. For this we will create a custom list called “Client Sites” with the following columns

Now we add some sites and assign them to the aforementioned clients. Note that in our example, ABC Incorporated has two sites. Since the Client column is a lookup against the Clients list we previously created, the “Client” column below is linked to its associated entry in the “Clients” list.

“IT Assets” List

Right! We have site information with some basic information about VPN details.How about we now create an IT Asset/Device list where we record the details about our equipment at these sites. In this list, we are interested in the type of device, its IP Address, serial number, purchase date and warranty.

Below is the configured list and some sample data

Examining the list relationships

So we now have three lists containing contact information, site information and device information. Any column that is of type “lookup” is automatically linked to the source of the lookup. So in the sample above of the device list, clicking on “Milton Keynes”, takes you to the entry in the “Client Sites” list as shown below.

From here, we can example the details of the client for this site, “ABC Incorporated”. Clicking on the client reveals our contact details for that client.

So what have we achieved so far? We have taken some useful information that would usually be stored in different locations (and potentially formats/systems) and linked it together. Being web based, the information is accessible from anywhere on the network. Remember, SharePoint also has a built in recycle bin and full version history if you wish to use it.

That is all well and good, but we have not linked this yet to our Cisco backups. In addition, we want to add our SOE documentation to be part of this also. So let’s create a new document library called “IT Asset Documentation”, and relate this to the devices.

IT Asset Documentation Library

So this time, we create a document library instead of a list. Here are our columns

So, now we create or upload our SOE, Network Diagrams and Maintenance Agreements so this document library as shown below.

In case you haven’t seen it before, Office 2007 is aware that you are using a SharePoint document library and allows you to set the values of these columns from within the application itself as shown below.

By creating the “Related Device” field, we now have a direct link to the three lists created earlier in this post. In addition, we have the ability to sort, filter and group documents in this library based on the columns we have added. So for example, here is another view of the document library grouped by document type.

For Cisco people reading this, notice how this view now looks a little like folders based on the document type? The importance and usefulness of views should not be underestimated. Unlike folders, views allow you to sort/manage your files much more flexibly than a static, folder structure. A single document library or list can have many views!

Finally, the configuration backups!

So if you were observant earlier, you would have noticed that I created a fourth document type called “Configuration Backup”, that we have not yet used. Let’s now integrate the TFTP server into this new library.

The simplest way that requires no customisation is to set the default document type to be “Configuration Backup”. That way, any file that is TFTP dumped into this library will be assigned that document type. So let’s modify the “Document Type” column and make this change.

It doesn’t really matter which TFTP method you use, so long as it is configured to drop files into this document library. If you use one of the TFTP servers used in part 2, then in this example the WEBDAV UNC version is:

• \\tidemo\tftp\IT Asset Documentation

And the URL for the web service/sdk method (as described in part 3 and part 4) is:

• http://tidemo/tftp/IT%20Asset%20Documentation/

I used the TFTPUtil method with web services and uploaded a file. Below is the document library with the new file. Note the document type!

Improving Integration with Workflow

Let’s now do a really simple workflow example to make this a little slicker. How about when a new configuration file has been TFTP’d into this library we perform these steps:

1. Check if the ‘related device’ field is empty
2. If it is empty, create a task for the network administrator to add the device details to the “IT Asset” list and link it to the uploaded file.

Since we have version control enabled in this library, the first time a file is uploaded the workflow will fire, and set this task for the network administrator. Once the related item is set, subsequent uploads will trigger the workflow, but it will do nothing because the related device has already been set.

I’ll create this workflow using SharePoint Designer. SharePoint designer workflows are by no means the answer to global warming but they are a great method for rapid workflow development with no programming expertise required.

In SharePoint designer we open our site and create a new workflow from the File Menu…

Name the workflow “Check Related-Device Field” or something similar and attach the workflow to the list “IT Asset Documentation”. Make sure the box to “Automatically start this workflow when a new item is created” is ticked.

Click NEXT and we add our first and only workflow step. The first thing to do is add a condition.

The first condition is always to compare the value of a field in the list the workflow is assigned to “IT Asset Documentation”. Click it, and choose the “Related Devices” field for the “field” parameter. Click the “equals” parameter and choose “is empty”.

Now we will create our action to assign the task to the network administrator. Click the “Actions” button and choose “Assign a To-do Item”.

Now in the newly created action, click “a to-do item” and on the next screen create a task name called “Update Cisco Config Documentation” and give it an appropriate description. Then Click “Finish”

Finally, we assign this to-do to the network administrator by clicking the “these users” link and finding the network administrator in the address book (Active Directory).

So here is the completed action.

Click “Finish” and we are all set!

Testing it out

So, let’s test by uploading a new file.

C:\>tftp 192.168.134.129 put c:\myfile.txt ABCRTR03
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

And the document library…

WOHOO! You can see a new column has been created to track our workflow status.

The status is “In Progress”, because it is waiting for a task to be completed. If we examine our SharePoint site, we have a new list, called … wait for it … “Tasks”!!

Let’s take a look at it!

If you click on this task (which also can be viewed and edited in Outlook), we can view this exciting task.

So our network administrator has received this task and has dutifully created a new entry for this router in the “IT Asset” list. They have then updated the “Related Device” column in the “IT Asset Documentation” as shown below

So now our intrepid network admin can edit this task and mark it as complete! Click da button!!

Now that the task has been completed, the workflow has completed and this is reflected in the “IT Asset Documentation” library as shown below. Note the workflow status is “completed”. Cool!

Conclusion

So what have we achieved here? This is a really basic example of improving configuration management, streamlining a process, improving the quality of documentation and enforcing accountability because we now have an audit trail of the task being assigned and completed.

We really are scratching the surface though. We have the tools here to implement much more that just making sure that something is documented.  In the next post, we will expand on this site and incorporate a change control mechanism using forms services. We will also start to look at this sort of stuff from a higher level, strategic viewpoint in the context of the ‘greater’ IT infrastructure management.

In conclusion, I hope that you Cisco guys found this post useful and look out for part 6 of this series in the near future.

Cheers

Paul

                      

No Tags

Welcome to part 4 of my series on demonstrating SharePoint’s usefulness for storing Cisco configuration backups. What a hard slog it’s been! The last article (part 3) of this series focused on how to modify an open source C# TFTP server to upload files into a SharePoint document library using the SharePoint SDK.

Here is the quick recap on what we have covered so far

• Part 1 illustrated how it it possible to use SNMP to tell a Cisco IOS device to dump its configuration to TFTP. We talked about the version control feature of SharePoint and why it makes sense to TFTP your configs to a SharePoint document library. We covered the WEBDAV network provider supplied with XP and Win2003 and finished off with a basic example using the TFTP server TFTPD32.
• Part 2 went into more detail about the issues you can face when using the WEBDAV network provider. It also went into more detail on 3 TFTP server products (WinAgents TFTP, SolarWinds TFTP Server and TFTPD32 and why TFTPD32 ended up being the best choice for this purpose.
• Part 3 then looked at a wonderful open source TFTP server written in C# called TFTPUtil. We modified the source code of this TFTP server to use the SharePoint SDK and upload files to a SharePoint document library.

Now both the WEBDAV and SDK methods had some issues. The WEBDAV method was obviously easy to set up because pretty much any application (theoretically anyway) can be made to work with it. I, however, had reliability issues with this method as part 2 detailed. The SDK method was much more reliable, but had its own problems. Many people would be uncomfortable with having to perform custom modification of an existing open source TFTP server, but more importantly, there are security implications with this method too.

So we have one remaining method that we can explore. This method is still based around modifications to the TFTPUtil source code but instead of using the SharePoint SDK, we instead use the SharePoint web services.

SharePoint 2007 Web Services

SharePoint can be remotely interacted with via Web Services. Out of the box, a number of web services are provided and shown below (along with the URL of the service based around my sample SharePoint server used in this series).

• Administration Web Service (http://tidemo/_vti_adm/Admin.asmx)
• Alerts Web Service (http://tidemo/_vti_bin/Alerts.asmx)
• Authentication Web Service (http://tidemo/_vti_bin/Authentication.asmx)
• Copy Web Service (http://tidemo/_vti_bin/Copy.asmx)
• Document Workspace Web Service (http://tidemo/_vti_bin/Dws.asmx)
• Forms Web Service (http://tidemo/_vti_bin/Forms.asmx)
• Imaging Web Service (http://tidemo/_vti_bin/Imaging.asmx)
• List Data Retrieval Web Service (http://tidemo/_vti_bin/DspSts.asmx)
• Lists Web Service (http://tidemo/_vti_bin/Lists.asmx)
• Meetings Web Service (http://tidemo/_vti_bin/Meetings.asmx)
• People Web Service (http://tidemo/_vti_bin/People.asmx)
• Permissions Web Service (http://tidemo/_vti_bin/Permissions.asmx)
• SharePoint Directory Management Web Service (
• Site Data Web Service (http://tidemo/_vti_bin/SiteData.asmx)
• Sites Web Service (http://tidemo/_vti_bin/Sites.asmx)
• Search Web Service (http://tidemo/_vti_bin/spsearch.asmx)
• Users and Groups Web Service (http://tidemo/_vti_bin/usergroup.asmx)
• Versions Web Service (http://tidemo/_vti_bin/Versions.asmx)
• Views Web Service (http://tidemo/_vti_bin/Views.asmx)
• Web Part Pages Web Service (http://tidemo/_vti_bin/WebPartPages.asmx)
• Webs Web Service (http://tidemo/_vti_bin/Webs.asmx)

So let’s go back to our TFTPutil project in Visual Studio and use Web Services instead of the SDK

Modification Example #2 Web Services

Initially I thought that I was going to have to write my own web service to upload a file to SharePoint, but like TFTPUtil, it didn’t take much googling to find that the hard work had already been done.

In this case, it was the excellent work by txs8311. He had written some articles on how to upload into SharePoint via methods provided by the out of the box web services. We will incorporate his work into TFTPUtil and see how it goes!

Out of respect for txs8311, I will not re-paste his code here, but I will show how I have incorporated it into TFTPUtil.

If you check his blog post, he has created a class called DocLibHelper. We will add this as a class to TFTPUtil and then call it from the TFTPServerProcess class that we modified in my last post.

So let’s go back to our TFTPUtil project in Visual Studio and add a new item to the project. Add a class and call it DocLibHelper.

\

Once added, paste the content of the DocLibHelper class from txs8311′s site. Do not forget to add the references to the libraries used by the code.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;
using System.Xml;

Finally, the code interacts with the Lists web service (lists.asmx) and we need to add a reference to a SharePoint web service so we can compile and test the code against the lists web service. From the “Project” menu, choose “Add Web Reference”. In the URL, connect to the lists service for your SharePoint server (in my case http://tidemo/_vti_bin/Lists.asmx)

You MUST name this Web Reference ListsService as txs8311′s code expects it to be this name.

That’s it for the webservice code! Now let’s modify the TFTPServerProcess class to use it! (I go into detail on modifying this class in the previous post).

State.Close();
if (IsSharePoint){
    // Now we add this newly downloaded file to the document library
    try
    {
        DocLibHelper docLibHelper = new DocLibHelper();
        Dictionary<string, object> properties = new Dictionary<string, object>();
        properties.Add("Title", State.Filename);
        string uploadURL = TempPath + @"/" + State.Filename;
        //Create or overwrite text file test.txt in 'Docs' document library creating folder 'Test Folder' as required.
        docLibHelper.Upload(uploadURL, System.Text.Encoding.ASCII.GetBytes(FullPath + @"/" + State.Filename), properties);

        // clean up temp file
        File.Delete(FullPath + @"/" + State.Filename);
    }
    catch (Exception ex)
    {
        AddMsg(Level.Verbose, "Problem opening SharePoint site" + TempPath);
        AddMsg(Level.Debug, "Exception: " + ex.Message + "\n" + ex.StackTrace);
    }
}
// This is the last block so go ahead and setup to delete stateStopListener(); 
StopListener(); 

So let’s have a closer look at this code. Now we are invoking an object based on txs8311′s the DocLibHelper class, rather than the SharePoint SDK classes. After creating an instance of DocLibHelper, we create a dictionary object to hold the properties of the file. We will write the name of the file to the “Title” column (the result of which you will see later).

We then call the Upload method, passing in the URL, a reference to the file to be uploaded and the properties dictionary.

One little gotcha!

One minor thing that I have to mention here, as I got caught by it. Web Services are case-sensitive! The invocation code on the console application was the following:

TFTPServer myTFTP = new TFTPServer();
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.Path = @"http://tidemo/tftp/backups";
myTFTP.StartListener(); 

But in SharePoint the document library is actually called “Backups”. See the problem? I kept getting exception errors telling me “List not found”, and it took me a while to twig to the fact that I had to fix up line 4 of the above code.

myTFTP.Path = @"http://tidemo/tftp/Backups";

Now that oughta do it! Let’s test!

Now we attempt an upload.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Sweet!

Let’s check the document library. First up view properties so we can see the title field. Very good.

Okay now let’s do a few more TFTP PUT’s.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip]

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

So let’s look at the version history. What do you see now? SharePoint will report changes to properties (column values) in the version history. Version 285 of myfile.txt was the version where I updated the title property.

This method works nicely. The only problem I encountered was a delay after the first running of the program. To connect to the lists webservice, authenticate and call the necessary methods a little while on the first attempt. But subsequent attempts were fine. Since I did this on a VM on a heavily loaded laptop, it may not be noticeable in a prod environment at all.

Modification Example #3, FrontPage Extensions and RPC

txs8311 also wrote a second part to his posts on uploading files into SharePoint. In this post he rewrote his DocLibHelper class to use FrontPage extensions and RPC. I did not try this, as I was satisfied with the modification Example #2. But all you should need to do, is replace the code in the DocLibHelper class with the code from his page.

If you are more keen than me, feel free to give it a shot and let me know how you go!

Summary of Upload Methods

So let’s recap on the three TFTP upload methods that we have used and summarise the advantages and disadvantages of each.

CleverWorkArounds Rating: Rock on SharePoint Web Services!

Additional Coolness

Since we have modified this TFTP Server to our requirements, you are pretty much free to add SharePoint specific features to make integration even nicer. One feature that springs to mind is setting more metadata properties when the files are uploaded. Here is an example. Let’s assume that we have a list of Cisco IP addresses and the MODEL and TYPE of device on that address. E.g 6509 Switch, 3725 Router or Aironet 350 Wireless Access Point.

It would be pretty easy to modify our TFTP server to be able to examine this list when receiving a file and based on the IP of the sender of the file, write the type of device to a “Device Type” column in our configuration backups library.

Better still, it would then be very cool if that list of IP’s and device types already exist in SharePoint. Imagine this list exists in the form of an IP address register or better still, an IT device asset register (along with serial numbers, maintenance contract information and the like). It would be easy for staff to maintain this information and the list would have version control just like our “Backups” document library. But for me what would be really nice about this method is the organisational efficiency that would be gained.

That gain is this: The person that updates the asset register list isn’t necessarily the Cisco engineer that sets up the device. Critical device information is entered ONCE and leveraged from then on. The next post in this series will expand on these benefits.

Conclusion

Right! We are DONE with getting files into SharePoint and about damn time too!

The next posts in this series will move away from application development and infrastructure issues and move into some higher level governance issues. The aim here is to show Cisco and IT infrastructure people what you can do with SharePoint once you get the data into SharePoint. I hope if I do a decent job of writing it, that it will be easy to see just how this can be re-used for any IT infrastructure management.

But for now, I bid you adieu until then….

                      

No Tags

As I write this series, it is getting less and less about Cisco and more and more about SharePoint. This article is definitely developer centric, but since Cisco guys tend to be interested in the guts of the detail, I decided to keep going .

If you read my articles I tend to take the piss out of IT role stereotypes just to make it more entertaining reading. Sales guys and IT Managers tend to cop it the most, but I also like to have a dig at the expense of the nerds too. Cisco nerds on the whole are a great bunch, but I have to say, the scariest nerd I have ever met drank Cisco kool-aid in jumbo size!

If you have gotten to this article after reading the first two and you are scoffing at my audacity to suggest you TFTP your configs into SharePoint, chances are most people think you’re scary! If you are hitting this series of articles for the first time, go back and read part 1 and part 2 before being scary!

Seriously now, I thought that this would be a 2 part set of articles, but I got all bogged down in the methods of getting files into SharePoint. The WEBDAV based methods described in the previous article is easy to do, but ultimately is not the recommended method. So now, we will look at the ‘proper’ ways to do it and see if they are worth the effort. They work okay, but are more complex and I’m not convinced that the governance issues are necessarily worth it for many readers.

Degree of difficulty for this article is varied.

CleverWorkArounds Coffee requirement rating (for an application developer):  

CleverWorkArounds Coffee requirement rating (for a non developer):    

How can we we *not* use WEBDAV?

If we are using methods other than WEBDAV, then how are our existing applications supposed to work? I mean, all they know is the windows networking API. So, unless the other methods are implemented as a redirector (as described in more detail in the “WEBDAV Interlude” section of my first article), how can they use any other API?

The answer is, they can’t! So we are leaving the world of freeware and cheap TFTP servers and writing our own one!

“What? I’m no coder”, you say? That’s okay, nor am I… but stick with me anyway.

Okay, how about we modify someone else’s open source TFTP Server to work with SharePoint via non WEBDAV methods?

Fortunately for us, such a TFTP server does exist. In addition to being conveniently modifiable (got to love open source), it is multithreaded, very well written and easy to understand and extend.  Kudos to the author. If you write a few scripts here and there, it should be enough for this article.

So let’s take a closer look. If you already know visual studio you can skip the next section and move to the section entitled “Examining the Code”. Otherwise let’s keep going getting this TFTP Server compiled!

Getting TFTPUTIL compiled

Head on over to http://sourceforge.net/projects/tftputil and download the latest release. Crank out Visual Studio 2005 and take a look at it.

This server is implemented as a class, so you have to write your own little program to make use of this class by creating a TFTP server object. You do by adding a new project to your Visual Studio workspace. But before we do that, how about we deal with a pre-requisite component.

The TFTPUtil class makes use of a library of useful functions called NSpring. You will want to download this. Grab the nspring.dll, and then tell Visual Studio how to find it by choose “Add Reference” from the “Project” menu, . At the resulting dialog box, go to the “Browse” tab and navigate to where you downloaded nspring.dll. (Best to put into C:\WINDOWS\SYSTEM32 or use the same folder where the TFTPutil code lives).

Right, we are all set! You could compile and build TFTPUtil.DLL right now, but it would not do you a lot of good because it’s a class and does not run on its own. It needs to be *invoked*. This means we need to create a little application that creates an *instance* of the TFTPUtil class. Right click on the project and choose “Add New project” as shown below.

For our purposes, we create a new “Console Application” project and call it myTFTPServer or something meaningful to you. This will be a small EXE file that uses the TFTPUtil class to create a TFTP server.

Now, right click on the newly created project and choose “Set up Startup Project”

Next step, we need to tell this new console application project how to find the TFTPUtil class. From the Project menu, choose “Add Reference”. At the resulting dialog box, go to the “Project” tab and you will see the open TFTPUtil project.

Right, that should be it! Your Visual studio should look something similar to this

Creating a basic TFTP server

This is a well written TFTP server. Everything is very nicely modular and easy to understand without getting into too much complexity.

So first up, here is the 5 lines of code of your console application to get the TFTP server working. Add this to the code section of the static void Main(string[] args) method.

using TFTPUtil;

TFTPServer myTFTP = new TFTPServer();
myTFTP.Path= "C:\TEMP";
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.StartListener(); 

Those 5 lines create a new TFTP server object, set the destination folder, allow it to receive files, overwrite received files and start the TFTP server. If you now compile and build this solution you will end up with the files TFTPutil.DLL and myTFTPServer.EXE.

You should be able to test it now. (By default, files will be uploaded to the folder the EXE is run from unless you specify a path).

Examining the code

Now let’s look at the area of the TFTP server where we need to add some code. It is in TFTPServerProcess.css in a method called “ProcessDataGram”

If you examine this method, you will see the following code (just search for it):

AddMsg(Level.Info, "Successfully received file " + State.Filename + " (" + State.Filesize.ToString() + " bytes) from " + State.RemoteIPAddress + " in " + TotalTime.ToString() + " seconds (" + (Rate).ToString() + " bytes/sec).");
//This is the last block so go ahead and setup to delete state
State.Close(); 

We will make some fairly basic modifications to the server in this section, so that the file is uploaded to SharePoint.

There are several methods that can be used to get files into SharePoint. We will do this first by using the most obvious way: The SharePoint SDK.

Modification example #1 – The SharePoint SDK

The SharePoint SDK is definitely the easiest way to upload a file, but there are several caveats:

• The TFTP server MUST run on a server that is a member of the SharePoint farm
• The TFTP service account MUST have administrative privileges on the server and SQL access

That second requirement in particular just plain sucks. After going to the trouble in part 1 to set up low privileged account “tftp service” with “contribute” access to a document library, we now find that we need to grant the account significantly more privileges than before. My security nazi Cisco readers will read that and say “no way will I do that”, and I would concur. But fortunately for my readers (and unfortunately for me) I only discovered that caveat AFTER I had gotten it working. So I will show you the code changes anyway.

So, first up I will tell you what we will do. The aforementioned ProcessDataGram uses a class called State that is used to write the file to the file system. Then later the method Close() closes the file that has been uploaded.

So what we will do, is upload the file to SharePoint just after State.Close(). This means the file is still downloaded to the local file system and then copied to SharePoint. Therefore, the local file system is now simply a temporary place for the file to be stored, prior to it be uploaded to SharePoint.

To do this, we will change how TFTPUTIL handles the PATH property. Recall in my basic TFTP console application example code I could set the path via:

myTFTP.Path = "C:\TEMP";

So let’s change this behaviour so that if Path contains the string “http://”, we assume it is a SharePoint path, rather than a file system path. Therefore we set the internal file path to %TEMP% so we can still capture the file, but then we will upload it to the http:// specified SharePoint site.

Path is a property of the TFTPUtil class that is defined in TFTPServer.cs. We will make a minor change to the Path property, because it performs a check that will always fail when a SharePoint URL is supplied.

public string Path
{
    get
    {
        return FullPath;
    }
    set
    {
        if (value.Contains("http://")) {
            FullPath = value;
        } else
        {
        if (System.IO.Directory.Exists(value))
            FullPath = value;
        }
    }
} 

Now we turn our attention to the TFTPServerProcess.cs file where we have to modify the TFTPServerProcess class. We will add two more properties to the TFTPServerProcess class. Find the declaration for FullPath and then add the two lines in bold below it.

private string FullPath = System.IO.Directory.GetCurrentDirectory();
private string TempPath = System.IO.Directory.GetCurrentDirectory();
private bool IsSharePoint = false; // If this is a sharepoint site 

Also while we are here, add a reference to the SharePoint dll.

using Microsoft.SharePoint; 

You will find the Microsoft.SharePoint.DLL in C:\Program Files\Common Files\Microsoft Shared\web server extensions on most installations.

Next, we need to add some basic logic to the code where the TFTPServerProcess class is initialised. It is an overloaded class so it is declared twice. To keep it simple, I have added the code to each declaration. If this is a Sharepoint path, it copies the supplied path to a variable called TempPath, and then puts the path of %TEMP% into the FullPath. This is deliberately done so that minimal code changes elsewhere are required (ie the rest of the class expects a variable called FullPath to point to a filesystem, not SharePoint).

Thus, it looks like this (bold):

Declaration 1:

public TFTPServerProcess() 

if (FullPath.Contains("http://"))
{
    IsSharePoint = true;
    TempPath = FullPath;
    FullPath = Path.GetTempPath();
}

Declaration 2:

public TFTPServerProcess(string FilePath,
    Level LoggingLevel,
    Level DisplayLevel,
    bool AllowRRQ,
    bool AllowWRQ,
    bool AllowWRQOverwrite,
    bool AllowTFTPOptions,
    bool CheckReadWriteState,
    int ResendInterval,
    int TimeoutInterval,
    Logger logger,
    IPAddress srvaddr)
{ 

    if (FilePath.Contains("http://"))
    {
        IsSharePoint = true;
        TempPath = FilePath;
        FullPath = Path.GetTempPath();
    }
    else
    {
        FullPath = FilePath;
    }

Finally, we add some code just after the State.Close() method as shown earlier. This takes the downloaded file from %TEMP%, uploads it to SharePoint and then deletes the file from %TEMP%.

State.Close();
if (IsSharePoint)
{
    // Now we add this newly downloaded file to the document library
    try
    {
        SPSite DestSite = new SPSite(TempPath);
        SPWeb DestWeb = DestSite.OpenWeb();
        DestWeb.AllowUnsafeUpdates = true;
        SPFileCollection files = DestWeb.Files;
        FileStream destFile = File.OpenRead(FullPath + @"/" + State.Filename);
        string destFileURL = TempPath + "/" + State.Filename;
        Hashtable MetaDataTable = new Hashtable();
        SPFile destSPFile = files.Add(destFileURL, destFile, MetaDataTable, true);
        destFile.Close();
        // clean up temp file
        File.Delete(FullPath + @"/" + State.Filename);
    }
    catch (Exception ex)
    {
        AddMsg(Level.Verbose, "Problem opening SharePoint site" + TempPath);
        AddMsg(Level.Debug, "Exception: " + ex.Message + "\n" + ex.StackTrace);
    }
}
//This is the last block so go ahead and setup to delete state
StopListener(); 

Note the HashTable MetaDataTable line. Even though we are not setting any metadata values, I have left in HashTable code because it illustrates how this method allows you to code in the setting of metadata columns in a document library. For example, imagine that the SharePoint document library has a column called “DeviceType”. Your code could say, write the string “Cisco” or “Linksys” to the DeviceType column when the file is uploaded.

So let’s try it out shall we?

Modify your console application to fix the path variable to a SharePoint doc library and then build the solution. 

TFTPServer myTFTP = new TFTPServer();
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.Path = "http://tidemo/tftp/backups";
myTFTP.StartListener(); 

Test the solution and you get a fairly unexciting console window.

Now we attempt an upload.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip it all working swimmingly]

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Also, this time around, none of the locking/access errors seen with the WEBDAV based methods – WOHOO!. Running this as a service also works fine, using the same freeware utility I used in the last post (ServiceInstaller ). Just compile the solution and then use ServiceInstaller as I described for TFTPD32 in the previous post.

Remember what I said before though, the “tftp service” account MUST have the following privileges on the server:

• The user is a server farm administrator.
• The user has Read and Write permissions on the content database.
• The user is a site collection administrator.
• The user has permissions to access the Windows SharePoint Services site or the SharePoint Server 2007 site through which the code iterates.

Read more at KB Article 935751, but the fundamental issue is that we are not running this code as the IIS process, but rather a separate user account. “If code is executed from a custom Windows-based application or from a custom console application, the code path does not use the Web context that is hosted by the Microsoft Internet Information Services (IIS) application pool. Therefore, specific user permissions are required in SharePoint Server 2007 and in the back-end data source”

Conclusion

The requirement for administrator level privileges sucks. It is not a showstopper, but it sucks nonetheless. So the next post, we will turn our attention to the Webservices methods of uploading data and see if they are any better. Then (finally!) we will get onto more interesting topics like making use of workflow and integrating it with other useful information.

If you are wondering whether you can download my changes and test for yourself, not yet . I’m going to contact the author of the TFTPUtil project first, and I have to perform more debugging . So right now you will have to do it yourself.

Thanks for reading.

Paul

                      

No Tags

Here I am back again, illustrating some of the interesting possibilities that SharePoint offers for Cisco people.

To recap my last post, I showed you a little perl script I wrote to get an IOS router or switch to dump its current configuration to a TFTP server. I then used one of several freeware TFTP servers to show how you can have a TFTP server save the captured file into a version enabled document library.

I then hit a snag in relation to using a Windows Service to do this task. In this article we will delve into this issue in more detail. In addition, I ended up delving much deeper than I intended. So, like my branding series, this is going to turn into a multi-part series too, covering some application development, configuration, security and governance issues. How many parts it will end up being is anybody’s guess!

This is a technically oriented series of articles for the most part, so for you people who like the governance and finance stuff, you may not get too much out of this one. Although this article (part 2) focuses on my issues and observations with the Windows WEBDAV client, if you are one of these people who have ‘special’ feelings when you see those pretty blue Cisco boxes like the image above, then you may find some useful content here.

SharePoint developers and architects may also find this of interest.

Now I previously mentioned that I tried three TFTP servers in writing this article and had some problems. They were:

• WinAgents TFTP (Commercial)
• SolarWinds TFTP Server (Freeware – Registration required)
• TFTPD32 TFTP Server (Freeware)

And let’s not forget good old PUMPKIN, just to make one of my CCIE friends, who read the first post and suggested I use it as an alternative, happy (although he forgets that years ago I was the one who told him about Pumpkin in the first place )

Let’s review their capabilities and cover the issues encountered with each. As we go, we will get to see the uglier side of WEBDAV.

WinAgents TFTP

I’ve used this product several times in the past. At the time it was pretty much the only product that would natively run as a Windows service. It is not free, but buying it won’t exactly break the bank. It has a nice GUI, good logging and the ability to set up virtual folders to different underlying locations. Below is the main configuration screen and the screen where multiple locations can be configured.

You would think that the above ability to virtualise folders would make this product an obvious choice for front-ending to SharePoint, because it could co-exist with native filesystem, DFS and 3rd party SMB based resources like those provided by samba.

You would be right, but the latest version is let down by one stupid design flaw. When you send a file to a WinAgents TFTP server that already exists, it will delete the original file and then copy the new file to the virtual folder. It may on the surface seem no different to simply overwriting a file – but there is a big difference in a SharePoint site.

All files in a SharePoint document library have a unique ID. It is a built-in column and can be made visible by modifying a document library view as shown below.

By deleting the file in SharePoint, that file is now put into the recycle bin. When you upload a file of the same name, SharePoint treats it as a new file and allocates it a new ID. Therefore, there is no version history at all. Each upload is deleting the previous version and uploading a new file.

This obviously blows away any potential to use WinAgents TFTP as a SharePoint integrated TFTP server .

CleverWorkarounds Rating: They probably thought it was a good idea at the time.

Solarwinds

So instead, let’s now turn our attention to the SolarWinds TFTP product. This one is free and also very easy to use. Like the WinAgents product, it can be run as a service out of the box, but it is not as comprehensive as its commercial competitor in terms of logging, granular security filtering and folder virtualisation.

Unlike its commercial cousin, however, it does not delete files before uploading, and therefore plays nice with SharePoint version control. That, combined with natively running as a windows service, makes it the obvious contender to be the SharePoint TFTP server of choice.

The GUI is basic yet functional. There is a status screen and a configuration screen that allows you to set the TFTP Root directory as shown below.

In the screen-shot above, you will notice that the TFTP Server root directory is \\tidemo\tftp\backups. As I explained in the first post, this is the WEBDAV UNC equivalent of the SharePoint site http://tidemo/tftp and the document library called backups.

If you recall in the first post, we created a user account called ‘tftp service’. This account has been granted access to the http://tidemo/tftp/backups library. So, we need to set the credentials of the WinAgents TFTP service to run as that user. Therefore, when files are received by TFTP clients, the server will be able to write to the library. Below is my screen capture of my configured TFTP server.

There was one other task I also had to do to test this setup. SolarWinds is a little bit naughty in that it logs to its installation folder. (‘tsk ‘tsk will never get a designed for XP gold logo). Being good practitioners of IT security, the ‘tftp service’ account is a domain user with no system privileges, and only has READ access to this folder by default. Changing configuration parameters (such as the UNC TFTP root) also would not be saved, because the TFTP server would not be able to write to its own configuration file when running as the ‘tftp service’ account..

So the permissions of the SolarWinds installation folder has been modified so that the ‘tftp service’ account has WRITE access as shown below.

So I started the service fine, and ran a test! Wohoo we have our file!!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

So I tried it again a few times and hit a snag after some successful retries…

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\TFTP 192.168.134.129 put myfile.txt
Error on server : The process cannot access the file ‘\\tidemo\tftp\backups\myfile.txt’ because it is being used by another process.
Connect request failed

This was odd. Suddenly I was unable to copy a new version of the file to TFTP. I was running many repeat tests, so perhaps there was some sort of temporary file lock causing a problem? So I waited for a bit and then was able to transfer successfully again as shown below.

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

But it wasn’t long before I got another and more severe error.

C:\TFTP 192.168.134.129 put c:\myfile.txt
Error on server : The network path was not found.

Connect request failed

The TFTP server log showed this error and ominous stack trace…

2008-01-23 21:49:53,102 [6] WARN  TFTPServer.Service.TransmissionPut – Exception caught while trying to open file for reading.
System.IO.IOException: The network path was not found.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access)
   at TFTPServer.Service.TransmissionPut.Begin(RequestPacket firstPacket)

Network path not found? What the hell was going on here? It worked a minute ago!

Eventlogs came up blank, and I tried an IISRESET. Still no-go. Restarted the TFTP service and I was able to transfer again. However, it didn’t take long for the problem to re-appear.

I tried all of the suggestions on Mark Muller’s blog article with no success.

I found two sites of interest and one seemed close to this issue. The first was here, where a developer was having the identical problem, and a Microsoft employee suggested not to rely on WEBDAV “especially when using services”. The second one was a confidence building quote from Microsoft themselves. They made an admission in their WEBDAV Whitepaper regarding the Web Client service. “The Web Client Service in Windows XP and Windows Server 2003 does not function correctly if you stop it then start it again without restarting the computer. Although a bug has been opened on this problem, the changes required to fix it are too large for a hotfix or a service pack and have therefore been delayed until a future major release of Windows”

Are you freakin SERIOUS!?!

Anyway, I decided that before I go off on a Microsoft bashing rant, I’d better double check that this problem was with SolarWinds TFTP or the obviously rock solid and stable WEBDAV client.

CleverWorkAround Rating (SolarWinds TFTP) – Judgement reserved till later

TFTPD32 TFTP Server

TFTPD32 is more than just a TFTP server. It also provides DHCP, Syslog and SNTP services as part of the deal. Unlike the previous two TFTP servers, it does not run as a windows service out of the box, but it can be easily made to.

Cisco have also decided that this is their recommended TFTP server as well, not a bad endorsement, eh!

If we focus on the TFTP server functionality only, it is basically the same as SolarWinds without the windows service bit. So WinAgents still wins for overall ease of use and granularity but as discussed it was a waste of time with SharePoint.

Once again there is a main status screen, showing you current activity, as well as a configuration screen where you set the TFTP server root UNC path.

So given that we had all of the WEBDAV stability problems with SolarWinds, this TFTP server allows us to test two scenarios. Since it can be run as standalone, we can run it interactively as our ‘tftp service’ account and test it. Then, we can use a 3rd party tool to make it run as a Windows Service, and compare notes.

So first up, let’s run it interactively as rtftp service’ account. This is achieved by RIGHT clicking on the TFTPD32.EXE file and choosing RUN-AS from the menu options. When prompted, enter the credentials of the ‘tftp service’ account.

If you want absolute assurance that TFTPD32.EXE is being run by the ‘tftp user’ then look for the process in “task manager” and check the “User Name” column.

So, here we go!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Now occasionally I received the error below, but unlike when using SolarWinds, running as a windows service using ‘ttp service’ user credentials, it would always happen once and subsequent attempts would be fine.

C:\TFTP 192.168.134.129 put myfile.txt
Error on server : Access violation
Connect request failed

Just to prove that it was much more reliable, we now look at the version history in SharePoint for the file myfile.txt

CleverWorkArounds Rating: Not flawless, but reliable

So we have ascertained that running TFTPD32 works pretty well when run interactively as ‘tftp service’ . Now we have to run TFTPD32 as a service and compare

TFTPD32 as a windows service

There are many utilities out there to run a regular EXE file as a Windows Service. One of the most popular ones is SVRANY from the Windows 200x resource kit. However I used the freeware ServiceInstaller because it had a nice GUI

Using it is mind numbingly simple. Run the application and fill in the form as shown below and click the ‘Install’ button. Once installed, you can then change the service credentials to ‘tftp service’ the same way we did it for SolarWinds.

Start the service, and check in Task manager that TFTPD32.EXE is running. (When running as a service you will not see the status screen).

The moment of Truth!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Like when running interactively, I still received the occasional  ‘Access Violation’ error but always recovered. Again to prove that I ran lots of TFTP commands, the version history is now up to 204 .

CleverWorkArounds Ratings (TFTPD32 as a service) – The best of the WEBDAV TFTP bunch

CleverWorkArounds Ratings (SolarWInds) – Looks like it might be part your fault after all!

CleverWorkArounds Ratings (WEBDAV) – It’s definitely your fault!

Conclusion

So after these tests, it seems that running TFTPD32 as a Windows Service is no less reliable than running it interactively. Perhaps then there is an issue with the way that SolarWinds use the windows networking API’s to write files. SolarWinds is a .NET application and TFTPD32 isn’t. Therefore, they may well be using different API’s under the hood.

I wasn’t satisfied with all of this though. Although we have achieved our goal of TFTP to a version controlled document library, the whole unreliable WEBDAV thing left me with a bad taste and I decided to go down a different path. What more could I do, you ask? How about write my own TFTP server?

The next 2 posts in this series will cover getting TFTP files into SharePoint via *non* WEBDAV methods. We will examine their reliability and examine a little of SharePoint web services and the SDK.

The fifth article will leave the issues of getting the files *into* SharePoint, and instead focus on what we can do with them once they are there! (Workflows, metadata and the like).

The final article (I suspect it will be the final anyway but i won’t know ’til I write the next two), will tie it all together in terms of governance. Thus, it will be a little similar to the branding series where the last article was largely non technical in focus and audience.

I hope you’ve enjoyed this little journey and didn’t find it too pointless

                      

No Tags

Cisco nerds! This series is just for you! I know that you think you’re way too cool for collaborative portals, especially a Microsoft one at that. Instead you are more interested in delving into the IOS command line, to perform arcane arts such as debugging that OSPF route redistribution into BGP or getting off on planning and implementing a large scale DMVPN solution. Maybe you’re into QOS and VOIP and simply dig all of those DSCP-COS mappings, class and policy maps and the like.

Although packets, cells and frames are your world, *nix is cool, Nagios is your idea of a portal and anything remotely connected to Microsoft fills you with contempt and is beneath you right?

Well if this is you, I do understand your point of view because I was you once, but after some therapy, I’m now out of rehab and doing just fine!

Having Cisco/general networking expertise will help you with this article, so depending on who you are, the amount of caffeine required to follow this will vary:

CleverWorkArounds Coffee requirement rating (for a CCNP or CCIE):  

CleverWorkArounds Coffee requirement rating (for a non Cisco person or CCNA ):  

Okay Cisco gurus, if there is anything about SharePoint that you guys need to know about, it is the power of the document library.

Unlike a regular file system, a SharePoint document library has a powerful version control mechanism that can make your life easier, as well as play nice with the rest of your colleagues (whom you despise because they don’t understand or appreciate you ).

So I am first going to show you an example of how you can leverage SharePoint to do something quite useful with Cisco gear, and then in the next post of this series I will expand further into general IT infrastructure admin benefits.

The Cisco bit..

(Non Cisco people, your eyes will glaze over – but try anyway!)

Back around 2000/2001 when I was a full time network manager, I wrote a perl script that used SNMP to instruct a router to dump its configuration to a TFTP server. I had a cron job that ran this script daily, and it would tell all of our switches and routers to write their current configuration to TFTP. It was nothing special – many Cisco network professionals do the same thing.

(Before you ask, I *know* there are tools and new methods to do this nowadays, but back then the choices were more limited)

Anyway, here is a snippet of the script in question. I know it’s not great but it did the job…

use Net::SNMP;

($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localtime;
@Days = (“Sunday”, “Monday”, “Tuesday”, “Wednesday”, “Thursday”, “Friday”, “Saturday”);
$day = $Days[$wday];

# Now call the procedure to do the deed!
&backup_configs(“192.168.16.1″,”RWSNMPcommunity”);
&backup_configs(“192.168.16.2″,”RWSNMPcommunity”);

The above code simply grabbed the date and then called the main function, passing the IP of the router and the SNMP community string for Read/Write access. The routine below, used SNMP SET commands to tell the router the location of the TFTP server, the name of the configuration file being written, and then told the device to execute the request.

sub backup_configs {
    ($ipaddr, $cs) = @_;
    ($s, $error) = Net::SNMP->session( -hostname  => $ipaddr,
                                -community => $cs,
                -debug => ’1′ );

    $hnh = $s->get_request(“.1.3.6.1.2.1.1.5.0″);
    $hostname = $hnh->{“.1.3.6.1.2.1.1.5.0″};

    $filename = “\\router-configs\\” . $day . “\\” . $hostname . “.cfg”;
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.2.667″, INTEGER, 1);
    # Set the source to startup-config
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.3.667″, INTEGER, 4);
    # Set the destination to tftp
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.4.667″, INTEGER, 1);
    # Set TFTP IP Address
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.5.667″, IPADDRESS,”192.168.18.5″);
    # Set destination file name
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1x.1.6.667″, OCTET_STRING,”$filename”);
    # Set it off!
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.14.667″, INTEGER, 1);
    $done = false;
    $counter = 0;
    while ($done eq false) {
        $check = $s->get_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.10.667″);
        if ($check->{“.1.3.6.1.4.1.9.9.96.1.1.1.1.10.667″} == 3) {
            $done = true;
        }
        $counter++;
        if ($counter > 500) {
            $done = true;
        }
    }
    if ($counter > 500) {
        print “$date:A problem occured – it looks like the attempt failed\n”;
    }
    # Clear the config!\n”;
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.14.667″, INTEGER, 6);
    #close the snmp session
    $s->close;

}

Now, I know there is a lot of ugly SNMP stuff in my above code and many readers are thinking, WTF!? You can safely ignore most of the code. However there is one bit that I do want to bring your attention to.

    $hostname = $hnh->{“.1.3.6.1.2.1.1.5.0″};

    $filename = “\\router-configs\\” . $day . “\\” . $hostname . “.cfg”;

Looking closely at this, you can see that I am setting the filename based around the name of the Cisco device (retrieved from SNMP) and the day of the week the script is run ($day). This is because when I backed up the configuration of the router, I wanted to keep multiple versions. For simplicity, I opted to keep 7 days worth of configuration dumps. So by using the day of the week in the file path, I ended up with config backups like this.

• router-configs/monday/mycorerouter.cfg
• router-configs/monday/myedgerouter.cfg
• router-configs/tuesday/mycorerouter.cfg
• router-configs/tuesday/myedgerouter.cfg

So, provided the path router-configs/<day of the week> exists on my TFTP box, I end up with a single file for each cisco device in each day of the week directory.

Neat, eh?

Yeah it’s neat, so why integrate TFTP with SharePoint?

Version control and workflow, my skeptical Cisco friends, that’s why! The easiest way to explain is to dive right into an example. So let’s do that right now!

There are a few TFTP servers out there for Windows (yeah we are using Windows – get over it). I used to use WinAgents TFTP server. Unfortunately though in writing this article, the most recent version of WinAgents TFTP does not work with SharePoint the way I want it to (if you run an older version it is fine). I also tried using SolarWinds freeware (registration required) TFTP server as it runs as a windows service. Unfortunately, I had some stability problems with WEBDAV when running as a service (not SolarWinds fault) which I will explain in the next post, so I ended up using TFTPD32 with a few modifications. (TFTPD32 is recommended by Cisco anyway).

What we will do is the relatively simple process of telling the TFTP server to drop files into a SharePoint document library via WEBDAV.

For reference, the TFTP server does not have to be on the same server as SharePoint when using the WEBDAV method.

The TFTPD32 Server Installation is straightforward. Once installed however, there are three additional steps that you need to perform:

• Create or nominate a user account to run the TFTP Service.
• Modify TFTP32 to run as a WIndows Service – using the credentials of a user account that has permissions to the document library that the TFTP server will dump the files to.
• Set the TFTP root directory to the UNC path of the SharePoint document library

I am assuming that you already have a SharePoint or WSS server. On the SharePoint side, we need to create a site and a document library to hold our configuration backups, grant the TFTP server permission to it and then enable version control on that document library.

So in my example, the user account that is going to upload files to SharePoint is called “tftp service“. I will refer to this account frequently in the rest of this post.

Also in this example, the SharePoint site is http://tidemo/tftp and the document library is called “backups“

http://tidemo/tftp/backups

Configuring SharePoint

(SharePoint people, I am covering the basics here – skip if you know it already).

In SharePoint, create a new, blank site to host the TFTP server files. Being a bastion of originality, in the example below I have named the site TFTP.

Now that the site has been created, we create a document library to store the files for the TFTP server. I have called it “Backups” and chosen not to use a default document template.

The next step, having created this document library, is to turn on version control. This is performed by choosing “document library settings” from the “Actions” menu.

In the version settings screen, choose to create major versions.

Finally, we need to grant our user “tftp service” write access to this new document library. The simplest way to do this is add “tftp service” to the members’ group of the SharePoint site. This will grant them ‘contribute’ access to the “Backups” document library. (This permission setting is just for demonstration purposes. Adjust permissions according to your specific requirements)

Right! Our doc library is ready! If at this point you open the document library via Windows Explorer as shown below, you will see that the UNC equivalent path for this example document library is \\TIDEMO\TFTP\BACKUPS

If you do not see your standard windows explorer window here and a UNC path, stop here! We need to troubleshoot the windows WEBDAV service. It might pay for you to skip to the “WEBDAV interlude section coming up)

One thing that people often overlook with SharePoint is that document libraries have this “windows explorer” view. What is so significant about this, you ask? It is the fact that you can reach any SharePoint document library via a UNC path. You can even map a drive to a SharePoint library too. Thus the following document library:

http://tidemo/tftp/backups

is accessible via

\\tidemo\tftp\backups

As a result of this functionality, pretty much any application that deals with UNC and mapped drives (such as ROBOCOPY) can happily copy files from your PC to a SharePoint library (with a large caveat I will explain later).

For educational purposes, let’s look at how windows explorer view is done. (If you are not interested, skip to the next heading where we configure SolarWinds TFTP.)

How is this done? A WEBDAV interlude

Now SharePoint does not actually provide support for SMB which is what ‘regular’ shared file systems use. The protocol that SharePoint supports is WEBDAV which stands for “Web-based Distributed Authoring and Versioning”. So, how is it that we can use Windows explorer, mapped drives and UNC paths to get to SharePoint files?

In Windows XP and above, Microsoft supplied a new network provider called the Windows WebDav client. (Yes, XP and above – Windows 2000 is not going to work this way for you). Also, Samba people who are reading this and thinking “ooh I can use smbfs to mount a SharePoint library to *nix”, sorry it ain’t gonna work because it is not SMB.

By creating the Web Client Service as one of the built-in network providers, it is automatically queried whenever you make connection attempt to a network resource (i.e. opening Windows explorer and navigating to a mapped drive or UNC path). You can see the provider by following these steps:

• On the Start menu, right-click My Network Places and then click Properties.
• On the top menu bar, click Advanced and then click Advanced Settings.
• Click the Provider Order tab.

You will see a dialog box with your provider order listed. Note that the Web Client Network (WebDAV) is last in the list by default.

Microsoft have a detailed whitepaper on how Windows Explorer view is implemented, and it is well worth a read. (I’ve previously blogged on quirks with it before.)

TFTP to SharePoint

So, now we know that on XP and above, any application that uses the windows networking API (interpret that as refer to a mapped drive or UNC path), can get data into and out of SharePoint. So let’s run the TFTPD32 and set it to use our document library.

Configuring TFTPD32

First up, grab TFTP32 and install it onto your chosen server. After successful installation run TFTP32.EXE from the installation directory.

Click OPTIONS and now set the path to the Windows explorer view path (UNC) of your site and document library

That is pretty much it! Now let’s test it!

Unfortunately, I don’t have any spare Cisco gear handy as Uncle Ebay helped me dispose of them once I stopped doing networking. So instead, I will use the built in TFTP.EXE utility built into Windows. This proves that a file can be saved to SharePoint via TFTP.

It works like this.

TFTP <hostname or IP> GET|PUT <filename>

eg

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

The output of the above command shows “transfer successful”. So let’s go and have a peek in our document library!

Sweeeet, we have a file called “myfile”.

So let’s go back to TFTP and re-run the command a few times.

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many re-executes]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Now let’s take another gander in the document library. Still the one file there. However, let’s now examine the version history for the file and see if there are multiple versions?

Sweeet, we now see that there are 7 versions stored for this file!

Conclusion (and the catch…)

So imagine that every day, you use the perl script example above to instruct your networking gear to copy their configuration to TFTP. Now you can simply use a standard file name convention (such as <hostname>-config.txt) do not need to worry about folder structure or dates within filename to ensure that multiple versions are maintained. You can go into the version history for a particular router configuration file and retrieve the one that you need.

Of course, all of your other important documents such as your network diagrams, SOE documentation, maintenance and support agreement details can also be placed into SharePoint document libraries and many other benefits are derived in addition to version history, such as search, workflow and easy offline storage.

But that is for the next post

Now for the catch … and it’s a biggie!

We of course, want to run our TFTP server as a windows service, so that the TFTP server does not have to be manually invoked.

Try as I might, I was unable to get a reliable TFTP service writing to SharePoint document library, when the TFTP server is running as a Windows Service! This majorly sucked and I was an unhappy camper!!

I have concluded that the WEBDAV client in WIn2003 is flakey. In the next post of this series, I will outline the problems that I had and how I eventually was able to come up with a semi-clever workaround to make it all work.

Bye for now!

Paul

                      

No Tags

First up, let’s finish off a little theory to go with the first post.

Other WIFI Security Measures

WEP and the much improved WPA should not be considered alone. Security in general requires a holistic approach to determine what functionality the business requires, what techniques can be used to provide that functionality and then assurance of the security of those features. Below are two additional techniques regularly used in conjunction with WEP or WPA to improve the security of a WLAN.

MAC Address Filtering

A common feature available with wireless access devices is the means to restrict access to only wireless devices that have their network (MAC) address configured in a filter list. Therefore, any device that tries to authenticate with a MAC address not matching the list will be rejected.

This method is not foolproof. If traffic can be decrypted then a valid MAC address could be determined and used to access the network.

Separate VLAN/DMZ for Wireless LAN

Most medium to large networks utilise virtual LAN’s (VLAN’s) to separate IP networks into logical groupings for security, performance and management reasons. Most good quality layer 3 switches provide VLAN’s that allow you to define firewall rules or access lists to restrict what resources on the network that wireless networks clients can access.

For more protection, the wireless network can be placed behind a fully featured stateful firewall rather than a VLAN alone.

WPA Wireless Clients Sample Implementation

This section shows an example of implementing WPA Enterprise with PEAP. Three components are required.

• The supplicant (the WIFI client),
• The Authenticator (the Access Point) and;
• An Authentication Server (A Radius server such as Microsoft IAS Server).

In this example the Access Point is a Cisco Aironet 1310 Series Outdoor Access Point/Bridge1 running IOS 12.3(7)JA.

In addition, since we are using PEAP, we need a suitable certificate server and we have used Microsoft Certificate Services.

Since we are using Microsoft IAS for Radius, which integrates with Active Directory, we have created 2 new Active Directory groups called "Wireless Users" and "Wireless Computers" respectively. These groups will be used to determine what computers and users are allowed access to the WIFI network. This ensures a high level of granularity for IT staff to manage access.

Configuring IAS (RADIUS)

IAS is a free component that is supplied with Win2k/Win2003 but is not installed by default. It can be installed via the Control Panel->Add/Remove Programs/Windows Components applet.

Both IAS and the authenticating access point need to be configured to perform Radius authentication. Firstly, you need to register IAS in Active Directory, so that IAS policies can be used on Active Directory users and computers to govern access.

In the left hand pane of the IAS management console, select Internet Authentication Service (Local) and right click to select the Register Server in Active Directory option.

If the server is already registered in active directory, this message will be shown

Adding the Access Point as a Radius Client

In IAS, you create a new RADIUS client and enter a shared secret.

Adding the IAS Server as a Radius Server on the Access Point

On the access point, the detail of the IAS server is entered.

Next we need to configure the Access Point to accept WPA clients and authenticate them to Radius. To do this, we first set encryption to WPA/TKIP.

Next we set the SSID to use Open Authentication and Network/EAP authentication. Network EAP is the better of the two, but Cisco have noted that ‘Some non-Cisco Aironet client adapters do not perform 802.1x authentication to the access point/bridge unless you configure Open authentication with EAP.’

We have already configured the RADIUS server, so we just use the default list. Under Key Management, we set it to mandatory and choose WPA. Accounting can also be enabled, and log on and log off events will be sent to the RADIUS server.

Configuring a Digital Certificate for the IAS Server.

If the server running IAS is already part of Active Directory, and there is a Certificate Server on the Active Directory Domain, the IAS server will already have a computer certificate installed. This can be verified using the Certificate Services MMC Span-In.

Open the Local Computer certificate store, choose Personal/Certificates and you should find a certificate matching the name of this computer. For example:

If you do not have a computer certificate, you need to obtain one via a certificate authority. This is not covered in this document.

Configuring IAS Wireless Policy

At this point we now need to configure the policy on the Radius server so that when a wireless device connects, the conditions that allow it to join the network are defined. The policies set will be to ensure the computer and user id supplied are members of the "Wireless Users" and "Wireless Computers" active directory groups.

User Access

The first policy to be created is for the user accounts that will be connecting to the wireless network. The wireless connection type will be defined and then the appropriate active directory security group will be specified.

In IAS Management console, create a new Remote Access Policy, select "Set up a custom policy" and enter "WFIC Wireless User Access" as the Policy name.

Now we add the conditions required for this remote access policy

Select Client-IP-Address, Add the IP address of the Wireless Access Point and click Ok

Select Windows-Groups and then click Add. Enter the wireless user groups "Wireless Users" and "Wireless Computers" as described earlier and click OK

Now we have completed the conditions for using this wireless policy. Click Next and grant remote access permission.

Now we have to specify the PEAP settings. Click Profile

Select the Advanced tab, highlight the Framed-Protocol (RADIUS Standard PPP) attribute and click Remove

Select the Encryption tab

Select the Encryption tab and remove all tick boxes, leaving only the Strongest Encryption (MPPE 128-bit) option ticked

Select the Authentication tab

Select the Authentication tab and click EAP Methods

Click Add to add an EAP type

Select PEAP and click OK. Now Click Edit to modify the PEAP type

Ensure that the certificate is the correct and ensure that MSCHAP is set as the EAP type

Back on the Authentication tab remove all other tick box options then click OK

You may see a message asking if you would like to view the online help to follow and configure dial-in settings

Click No and Next to continue. The wizard will now tell you the profile is complete and summarise the conditions and profile settings.

Computer Policy

Another policy can optionally be created for Domain Computers. The policy steps are identical to the steps outlined in the user policy except during the setting of policy conditions, you use the Active Directory group that the wireless computers are a member of.

Granting Active Directory Rights

The last step is to ensure that the user and computer accounts in Active Directory have rights to ‘dial in’ – which is Microsoft’s term for using Radius Authentication.

A user account needs "Control Access through Remote Access Policy" set.

A computer account needs the same setting.

Wireless Client Configuration Settings

This section only deals with the configuration of Windows XP Clients

Digital Certificate

Using PEAP, the WIFI clients do not need to use a digital certificate. Only the server uses one to identify itself to the client. However, the clients need to trust the certificate authority (CA) that issued the server certificate. To do this, the certificate of the CA needs to be added to the trusted certificate list on the client.

If the computer is a member of the Active Directory Domain, the CA certificate is likely already installed. This can be checked using the same steps defined in the section "Configuring a Digital Certificate for the IAS Server" except you go to the "Trusted Root Certification Authorities" store to find the CA certificate.

If the certificate is not installed and if the CA certificate Using Microsoft Certificate Services, it is easy to obtain and install it. The certificate can be installed by double clicking on the certificate file stored on a shared folder, floppy or USB device. Alternatively, the client can request the CA certificate from the CA website.

Wireless Networking Configuration

In the properties of the wireless network adapter, add a new wireless network, specifying WPA and TKIP

On the authentication tab, choose PEAP as the EAP type

Click Properties and tick ‘validate server certificate’ and ensure the CA that issued the server certificate is selected.

If the certificate authority is not listed above, than this machine does not trust it and it needs to be added to the trust list as per section titled "Digital Certificate"

Ensure ERP-MSCHAPv2 is selected at the ‘select authentication method’ list and click ‘properties’.

This dialog box allows you to select if XP will attempt user authentication based on the currently logged in user, or prompt for credentials to be entered. If this workstation is a member of the domain, and the user logs into it via a domain account, this should be ticked.

Click OK and OK again to return to the Wireless network properties screen.

If a computer authentication policy has been chosen as per section entitled "Computer Policy", tick the "Authenticate as computer when computer information is available".

At this point the wireless client should be able to connect to the wireless network if the user and computer credentials are valid. The event logs on the IAS server now will show detailed records of all wireless network authentication attempts including username, date/time, IAS policy used and whether access was granted or denied.

Below is a sample IAS log entry from an access point using the IP address 192.168.100.26 and an IAS policy called "AP Test".

Event Type:    Information

Event Source:    IAS

Event Category:    None

Event ID:    1

Date:        20/01/2006

Time:        4:45:07 PM

User:        N/A

Computer:    MYSERVER

Description:

User DOMAIN\some guy was granted access.

Fully-Qualified-User-Name = domain.net/MY Company/DOMAIN/Some Guy

NAS-IP-Address = 192.168.100.26

NAS-Identifier = bridge

Client-Friendly-Name = AP Test

Client-IP-Address = 192.168.100.26

Calling-Station-Identifier = 0013.ff10.427c

NAS-Port-Type = Wireless – IEEE 802.11

NAS-Port = 282

Proxy-Policy-Name = Default Connection

Authentication-Provider = Windows

Authentication-Server = <undetermined>

Policy-Name = Test – Wireless

Authentication-Type = PEAP

EAP-Type = Secured password (EAP-MSCHAP v2)

                      

No Tags

Like my posts on IT governance standards, I produced this training material some time back when I was doing a lot of IT security work. I’ve since moved onto other IT disciplines, but I hope that this article is of some use to those looking for an introduction to WIFI security. I have divided the material into two parts. The first half is background and theory, and the second half of a practical example.

I remember when writing it, my audience, although technically savvy did not a have a strong background in cryptography or security. So I tried to make it easy to read, rather than too technical. Not sure if I succeeded!

A Security Primer – Security Principles

The ISC2 common body of knowledge (CBK) is the basis for the Certified Information Systems Security Professional (CISSP) certification.

The CBK defines 3 major security principles known as the CIA triad: Confidentiality, Availability and Integrity.

• Confidentiality is the principle that information will not be disclosed to unauthorized subjects. Unauthorized Network sniffing is an example of a violation of confidentiality.
• Integrity is trust that can be placed in the information. Data integrity is having assurance that the information has not been altered between its transmission and its reception. Data integrity can be compromised when information has been corrupted, willfully or accidentally, before it is read by its intended recipient.
• Availability defines that information or resources are available when required and are accurate, relevant and timely. It is certainly possible that a confidentiality and integrity are protected, but an attacker causes resources to become less available than required, or not available at all, eg Denial of Service.

Identification, Authentication, Authorisation and Accountability

• Identification describes a method of ensuring that a subject (user, program or process) is the entity it claims to be
• Authentication is the process of attempting to verify the digital identity of the sender of a communication such as a request to log in. The sender being authenticated may be a person using a computer, a computer itself or a computer program.
• Authorisation is the process of verifying that a known person has the authority to perform a certain operation.
• Accountability, is synonymous with non-repudiation. The non-repudiation of receipt of information means that an agent can’t deny receiving information. The non-repudiation of sourcing information means that an agent can’t deny sending information.

Encryption, Integrity and Authentication: WEP vs. WPA vs. WPA2

WEP

Everyone who has ever site up wireless will know WEP. They will also likely know that WEP is a ‘bad thing’. Here we explain why WEP is now considered to be a very poor choice for wireless security and what was done to fix it.

The original Wired Equivalent Privacy (WEP) algorithm was used to protect wireless communication from eavesdropping. It uses predefined ‘WEP keys’ to encrypt the traffic using the RC4 encryption algorithm. It also ensures data integrity by using an "Integrity Check Value". This is 4 bytes and appended to the end of each packet.

One of several security problems with WEP is that using a static, unchanging WEP key means that brute force efforts can be undertaken to ‘crack’ the key by capturing enough encrypted packets. WEP also uses an Initialisation Vector (IV) which is used to ensure that text encrypted with the same encryption key translates to a different ciphertext value. Unfortunately, IV is too small and is vulnerable to attacks that make the IV easy to determine. In addition, the size of the key – 40 bits – has been cited as a weakness of WEP. When the standard was written in 1997, 40-bit keys were considered reasonable for some applications. Since the goal was to protect against "casual eavesdropping" it seemed sufficient at the time. The U.S. did not tightly control exports of 40-bit encryption, and the IEEE wanted to ensure exportability of wireless devices.

The WEP Integrity Check Value (a fancy name for a hash) is based on CRC-32, an algorithm for detecting noise and common errors in transmission. CRC-32 is an excellent checksum for detecting errors, but a poor choice for a cryptographic hash. Better-designed encryption systems use algorithms such as MD5 or SHA-1 for their Integrity Check Values. The CRC-32 Integrity Check Value has a weakness that allows an attacker to modify an encrypted message and easily fix the Integrity Check Value so the message appears authentic.

So in relation to encryption and integrity, WEP fares rather badly. How does WEP fare in the areas of identification, authentication, authorisation and accountability? Not well either.

WEP defines two forms of authentication: Open System (no authentication) and Shared Key authentication. These are used to authenticate the wireless client to the access point. The idea was that authentication would be better than no authentication because the user has to prove knowledge of the shared WEP key, in effect, authenticating himself. In fact, the exact opposite is true: If you turn on authentication, you actually reduce the total security of your network and make it easier to guess your WEP key.

Shared Key authentication involves demonstrating the knowledge of the shared WEP key by encrypting a challenge to the access point, and the access point successfully decrypting that challenge. The problem is that a monitoring attacker can observe the challenge and the encrypted response. From those, he can determine the RC4 stream used to encrypt the response, and use that stream to encrypt any challenge he receives in the future. So by monitoring a successful authentication, the attacker can later forge an authentication.

So with open authentication, any wireless device that knows the WEP key will be accepted by the access point.

Hence, WEP is now considered to be a very poor choice for WIFI security. In response, the Wi-Fi Alliance, an industry trade group, created a standard called Wi-Fi Protected Access (WPA).

WPA

At the time, elements of WPA were actually taken from a work in progress IEEE standard called 802.11i. 802.11i was a new, in development standard aimed at a new generation of devices and had no backward compatibility requirements for WEP, meaning completely new wireless hardware would have to be produced to take advantage of it.

Thus, WPA in effect an interim measure designed to work with the constraints of WEP based systems, so that rather than a wholesale change/removal of wireless hardware, customers could upgrade software and firmware to gain the benefits of WPA. This was an attractive proposition from the point of view of the device manufacturers as it reduced redevelopment costs. But was it enough?

WPA offered a set of new features to alleviate the problems of WEP. A protocol called TKIP was added to WEP encryption. TKIP has been designed to ‘wrap’ around WEP and improve the main security issues.

• The size of the IV has been doubled to 48 bytes which is also added to the MAC address and original WEP key per frame.
• TKIP offers ‘re-keying’, where multiple encryption keys are changed on the fly at a configurable timeframe (defaults to per 10000 frames).

(WEP alone uses the same encryption key. Thus with WPA, even if you managed to determine an encryption key, you have a very limited time in which you can use it to decrypt data before it changes again.)

WPA also has replaced the ICV with the Message Integrity Code (MIC) which eliminates some of the weaknesses with ICV.

CISCO and CKIP

While the WPA and WPA2 standards were being agreed to, Cisco added an extra feature to WEP to improve one of its major weaknesses – that of poor confidentiality via encryption. This is called the CKIP protocol. CKIP is a Cisco proprietary protocol that essentially does the same thing as TKIP does in WPA. This was designed to alleviate the immediate encryption concerns with WEP. CKIP however was not widely adopted as most vendors (Cisco included) opted to use the WPA standard as the interim measure.

WPA2

WPA2 is simply the complete 801.11i standard that was referred to earlier. It is actually very similar to WPA, as WPA used much of this standard. The main difference is the inclusion of the AES encryption protocol, which is an alternative to TKIP/WEP encryption. To support AES you need specialised hardware and thus require a new generation of wireless devices to support.

Advanced Encryption Standard (AES), also known as Rijndael, is an encryoption algorithm adopted as an encryption standard by the US government and is designed to replace the widely used by aging Data Encryption Standard (DES)

Frame integrity is now handled by a protocol called CCMP (with the simple name of "Counter Mode with Cipher Block Chaining Message Authentication Code Protocol"). CCMP is built around AES.

802.1x / EAP Authentication

Encryption is only part of the WIFI security picture. Even with an unbreakable encryption algorithm, if a malicious user was able to authenticate to the access point, they would be able to participate on the wireless network like any other authorised wireless client as the access point would exchange encryption keys with this seemingly ‘trusted’ client.

WPA/WPA2 adopted IEEE 802.1x / EAP authentication (WPA Enterprise), rather than re-invent the wheel. 802.1x was originally designed as a port based network access control that ensured that a user could not access the network unless authorised. Note the word "user", whereas WEP could only offer "system" authentication.

EAP is a complex standard described in detail later. However the IEEE did recognise that EAP added complexity and thus, added pre-shared keys for authentication (WPA Personal).

Note: Pre shared keys are common in IPSEC/L2TP for authentication as well.

WPA Personal

When WPA Personal uses pre-shared keys (PSK) for authentication, a common password or passphrase is configured on each wireless device and the access point. This is used to authenticate the client with the access point. Once this takes place, encruption keys can be exchanged and traffic can flow between client and access point.

However, if you do not make your pre-shared key long, you are susceptible to an offline dictionary attack where an attacker grabs a few packets at the time a legitimate station joins the wireless network and then can take those packets and attempt to recover the PSK used.

In addition, management of the PSK as the network grows becomes a major issue. It needs manual configuration on all WIFI clients. Thus, changing the PSK would be time consuming and result in downtime.

WPA Enterprise

WPA adopted 802.1x / EAP for authentication. EAP is an authentication framework, not a specific authentication mechanism. The EAP provides some common functions and a negotiation of the desired authentication mechanism. Such mechanisms are called EAP methods and there are a myriad of options and standards available. This flexibility has the disadvantage of being complex and difficult to understand the intricacies of every option.

The table below is a summary of the most applicable options. They are not the exhaustive list, but the others are not listed because they are either not applicable or are missing required features.

LEAP

The Lightweight Extensible Authentication Protocol (LEAP) is a proprietary EAP implementation by Cisco Systems.

There is no native support for LEAP in any Windows operating system but is supported by third party supplicants (IEEE speak for client software). Support for other operating systems is unclear. This protocol is known to be vulnerable to dictionary attacks as it is based on shared secret passwords. Cisco still maintains that LEAP can be secure if sufficiently complex passwords are used.

EAP-TLS

EAP-TLS is an IETF open standard, and is well-supported among wireless vendors. It offers a good deal of security, due to using digital certificates and Public Key Infrastructure (PKI). Its main disadvantage is the management overhead to set up and maintain a PKI as well as client-side certificates. This factor has led to few deployments compared to other EAP solutions.

The requirement for a client-side certificate, however unpopular it may be, is what gives EAP-TLS its authentication strength and illustrates the classic convenience vs. security trade-off. A compromised password is not enough to break into EAP-TLS enabled systems because the hacker still needs to have the client-side certificate. When the client-side certificates are housed in smartcards, this offers the most secure authentication solution available because there is no way to steal a certificate from a smartcard without stealing the smartcard itself. Any physical theft of a smartcard would be immediately noticed and revoked and a new smartcard would be issued.

There are client and server implementations of it in Microsoft, Cisco, Apple, Linux, and open source. EAP-TLS is natively supported in MAC OS 10.3 and above, Windows 2000 SP4, Windows XP, Windows Mobile 2003 and above, and Windows CE 4.2.

EAP-TTLS and PEAP

EAP-TTLS and PEAP both do not use client side certificates. Instead they use PKI certificates only on the authentication server which is used to create a secure TLS tunnel to protect user authentication. PEAP was developed by Microsoft and Cisco and EAP-TTLS developed by Funk Software.

PEAP is natively supported in MAC OS 10.3 and above, Windows 2000 SP4, Windows XP, Windows Mobile 2003 and above, and Windows CE 4.2.

The server side implementation of PEAP/EAP-MSCHAPv2, called IAS (Internet Authentication Service), is also included in Windows 2003 server.

We will not discuss the differences between PEAP and EAP-TTLS here, except to say that PEAP has now overtaken EAP-TTLS in terms of popularity.

802.1x Components

802.1x consists of 3 main components.

• A client (supplicant)
• An ‘authenticator’ (access point)
• An ‘authentication server’ (RADIUS).

EAP is the mechanism used by all 3 components to communicate authentication information.

RADIUS Server

In a large wireless network, a means is required to provide central authentication for clients. A RADIUS server fills this need. The RADIUS protocol is the industry standard for AAA (Authentication, Authorisation and Accounting). Many Radius servers exist, and Microsoft provide a built in Radius Server with Windows Server called IAS which integrates with Active Directory. Other Radius servers can integrate with 3rd party authentication mechanisms (RSA keys and smartcards for example)

Bringing it all together

The key goal of any wireless network is to ensure confidentiality, integrity and availability. Without proper authentication, encryption is of limited value as it would be impossible to determine if the data has been tampered with.

802.1x / EAP provide the means to centrally manage who and what can access the wireless network. This also provides us with detailed accounting records of who or what has attempted to access the network (Accountability).

The mutual authentication offered by some EAP methods also means that a client can be sure that it is accessing the correct access point and not a malicious access point set up to lure clients into connecting to it and giving up sensitive information.

TLS and PEAP are the two best choices for wireless authentication. TLS offers more security than PEAP due to client certificates required as well as user credentials (two factor authentication). However PEAP is much easier to set up and deploy than TLS because you do not have to implement a PKI solution (which is a whole project in itself). In addition, when using an Active Directory integrated Radius server, authentication can be restricted to particular groups of users and computers and AD account lockout policies will be enforced.

If we combine this with the MAC address filtering and VLAN access lists described in section 2.1.4 we have a holistic solution that addresses the key security requirements.

We have:

• An encryption mechanism that changes keys frequently and is unique for each client, so that by the time you have brute forced a encryption key, it has already been discarded and a new key has been generated
• An authentication mechanism that requires the user to have a valid Active Directory user account, optionally a valid Active Directory Computer Account and optionally a valid client certificate issued by a trusted Certificate Authority (EAP/TLS)
• An authentication mechanism that ensures that the client can ensure that the access point it is connecting to is known and trusted via a server side digital certificate.
• An authentication mechanism that is centrally managed, so when users or computers are disabled they are unable to access all wireless services.
• Centralised location for logging of access requests and their outcomes
• Filtering so that only pre-defined MAC addresses can authenticate to an access point
• Access lists that ensure that the Wireless VLAN can only access authorised parts of the network.

In the next part to this post, we will discuss some additional security mechanisms for WIFI, as well as a WPA/PEAP example implementation.

                      

No Tags

Well, yeah you got me.. but a few years back I was a Cisco nerd in the ISP industry and got pretty handy at it. I wrote this article 3 years ago but then forgot about it until recently.. so it may as well see the light of day because at the time I wrote it, there was very little info out there on this subject.

Linksys to Cisco IOS IPSEC Tunnel

The network

Site 1:  (Linksys WAG54G)

• Inside Network: 172.18.30.0/24
• Gateway IP 200.100.1.1
• Inside IP 172.18.30.1
• NAT: Overloaded on Outside interface
• LinkSys WAG54G. Software Version: 1.02.9, Dec 22 2004Site2:

Site 2: (Cisco 3725)

• Inside Network: 192.168.100.0/24
• Gateway IP: 200.56.4.1
• Inside IP 192.168.100.1
• NAT: Overloaded on Outside interface
• Cisco 3725 Router. IOS. IOS ™ 3700 Software (C3725-IK9O3S-M), Version 12.3(6), RELEASE SOFTWARE (fc3)
  System image file is "slot0:c3725-ik9o3s-mz.123-6.bin"

The IPSEC Parameters

Probably the most common combination for IPSEC is to use 3DES/SHA/DH Group 2. For transforms ESP-3DES-SHA

• 3DES, SHA1, ESP (no AH).
• PFS is OFF!
• DH Group 2 (Group1 and 2 only supported on the linksys)

The IKE Parameters

• 3DES, SHA, DH Group 2
• All IP traffic to traverse VPN tunnel.

Linksys Configuration

• Security Tab – Choose VPN
• You do not need to enable IPSEC passthrough. You would only do this if another device behind the linksys was actually doing the IPSEC tunnel.
• Create a new tunnel entry and give it a name. Specify a subnet for the local security group (the local subnet) and set 172.18.30.0 with a 255.255.255.0 subnet mask
• Specify a subnet for the remote security group. 192.168.100.0/255.255.255.0

• Now we have to specify the IP address of the remote IPSEC peer. In our example we are using an IP address not a domain name

• Encryption should be set to 3DES and hash (authentication) to SHA. This is Phase II encryption (as will be confirmed in advanced settings screen). Set PFS to OFF in this example (Cisco defaults to off).  The Key Lifetime here actually is Phase II key lifetime. Phase 1 key lifetime is in the advanced screen.
• IKE is used, rather than manual.  Cisco default setting for Key Lifetime is 86400 seconds. So change this end to match. Set the shared key as well.
• Now click the advanced tab. Here we configure the phase 2 config. Repeat 3DES and SHA everywhere with 1024bit group (DH Group 2)

Note how this says that the linksys will try and offer DES/MD5/768, 3DES/SHA/1024 and 3DES/MD5/1024. In fact proposal 1 and 3 are therefore identical. I have not looked deeply into this, although it increases the Phase 1 work required when the two peers negotiate phase 1.

CIsco Configuration

Note that this particular router has 6 IPSEC tunnels to different peers. In this example, the connection to the WAG54G is the latest IPSEC tunnel. Comments are in itallics

SITEB#sh run
Building configuration…

First we define the ISAKMP policies that this router will accept. 4 are defined here. This is because the other 5 IPSEC tunnels have unique IKE requirements. Each different IPSEC peers have different capabilities.

crypto isakmp policy 1 (3DES/SHA/DH Group 5)
encr 3des
authentication pre-share
group 5
!
crypto isakmp policy 2 (3DES/MD5/DH Group 5)
encr 3des
hash md5
authentication pre-share
group 5
!
crypto isakmp policy 3 (DES/SHA/DH Group 5)
authentication pre-share
group 5
!
This is the ISAKMP policy that will match the LINKSYS router at Site A.
crypto isakmp policy 4 (3DES/SHA/DH Group 2)
encr 3des
authentication pre-share
group 2

Note that all ISAKMP use pre-shared keys for authentication. These are defined below.
crypto isakmp key xxxxxxxxx address yyyyyy
crypto isakmp key xxxxxxxxx address yyyyyy
crypto isakmp key xxxxxxxxx address yyyyyy
crypto isakmp key xxxxxxxxxx address yyyyyy
crypto isakmp key xxxxxxxxxx address yyyyyy
crypto isakmp key thisisasecretpassword address 200.100.1.1
!
!
This is the IPSEC Parameters. (Phase II) – explain here..

crypto ipsec transform-set Default esp-3des esp-sha-hmac
!
Now we define the crypto map itself. We specify the ACL that needs to be matched for each peer and then the transform set to use. Remember the first 5 crypto map entires are for other VPN’s.

crypto map IPSEC 5 ipsec-isakmp
set peer yyyyyy
set transform-set Default
match address xxxxxx
crypto map IPSEC 10 ipsec-isakmp
set peer yyyyyy
set transform-set Default
match address xxxxxx
crypto map IPSEC 15 ipsec-isakmp
set peer yyyyyy
set transform-set Default
set pfs group5
match address xxxxxx
crypto map IPSEC 30 ipsec-isakmp
set peer yyyyyy
set transform-set Default
match address xxxxxx
crypto map IPSEC 40 ipsec-isakmp
set peer yyyyyy
set transform-set Default
match address xxxxxx
crypto map IPSEC 50 ipsec-isakmp
set peer yyyyyy
set transform-set Default
match address xxxxxx
crypto map IPSEC 60 ipsec-isakmp
set peer 200.100.1.1
set transform-set Default
match address SITEB_to_SITEA
!
!
!
!
interface FastEthernet0/0
no ip address
shutdown
speed auto
full-duplex
!
Now we apply the crypto map to the externally facing interface.

interface FastEthernet0/1
ip address 200.56.4.1 255.255.255.252
ip access-group Firewall in
no ip redirects
ip nat outside
duplex auto
speed auto
no cdp enable
crypto map IPSEC
!
interface FastEthernet1/0
description Inside
ip address 192.168.100.1  255.255.255.0
ip nat inside
duplex auto
speed auto
!
ip nat inside source list inside_net interface FastEthernet0/1 overload
ip route 0.0.0.0 0.0.0.0 200.56.4.2
!
This is the ACL that the policy map called IPSEC uses to identify traffic to be IPSEC encrypted
ip access-list extended SITEB_to_SITEA
permit ip 192.168.100.0 0.0.0.255 172.18.30.0 0.0.0.255

ip access-list extended Firewall
remark
remark Allow TCP replys
remark
permit tcp any any established
remark
remark For VPN IPSec
remark
permit esp host 200.100.1.1 host 200.56.4.1
permit udp host 200.100.1.1 host 200.56.4.1 eq isakmp
permit icmp any any parameter-problem
permit icmp any any source-quench
permit icmp any any echo-reply
permit icmp any any time-exceeded
permit icmp any any unreachable
deny   ip any any log

This ACL is used to Network address translate all outgoing packets *except* the stuff for IPSEC. That has to stay with source address preserve. Therefore by using a deny entry we stop the router NATting any traffic from 192.168.100.0/24 to 172.18.30.0/24

ip access-list extended inside_net
deny   ip 192.168.100.0 0.0.0.255 172.18.30.0 0.0.0.255
permit ip 192.168.100.0 0.0.0.255 any
!
end

Example 1: Tunnel initiated from Linksys end

Logs (linksys). Linksys logs are quite basic, but you can see the 3 IKE Phase 1 packet exchange and tell that its using Main Mode (eg MM_I1) and then the Phase II Quick Mode (QM_I1) exchange.

Main mode Phase 1 happens first
2005-02-06 22:29:39 IKE[1] Tx >> MM_I1 : 200.56.4.1 SA
2005-02-06 22:29:39 IKE[1] Rx << MM_R1 : 200.56.4.1 SA
2005-02-06 22:29:39 IKE[1] ISAKMP SA CKI=[9ddfe293 4d8be14b] CKR=[cca2f07a 9045af27]
2005-02-06 22:29:39 IKE[1] ISAKMP SA 3DES / SHA / PreShared / MODP_1024 / 86400 sec (*86400 sec)
2005-02-06 22:29:39 IKE[1] Tx >> MM_I2 : 200.56.4.1 KE, NONCE
2005-02-06 22:29:40 IKE[1] Rx << MM_R2 : 200.56.4.1 KE, NONCE, VID, VID, VID, VID
2005-02-06 22:29:40 IKE[1] Tx >> MM_I3 : 200.56.4.1 ID, HASH
2005-02-06 22:29:41 IKE[1] Rx << MM_R3 : 200.56.4.1 ID, HASH
Great – we got this far. Once you see QM you know you have made it to Phase II
2005-02-06 22:29:41 IKE[1] Tx >> QM_I1 : 200.56.4.1 HASH, SA, NONCE, KE, ID, ID
2005-02-06 22:29:42 IKE[1] Rx << QM_R1 : 200.56.4.1 HASH, SA, NONCE, KE, ID, ID, NOTIFY
2005-02-06 22:29:42 IKE[1] Tx >> QM_I2 : 200.56.4.1 HASH
2005-02-06 22:29:42 IKE[1] ESP_SA 3DES / SHA / 86400 sec / SPI=[c243f13f:a213c248]
2005-02-06 22:29:42 IKE[1] Set up ESP tunnel with 200.56.4.1 Success !
2005-02-06 22:29:42

Logs (Cisco). Below is the output from the commands Debug crypto ipsec and debug crypto isakmp

Now for Cisco logs there is a massive amount of detail for better or worse , So we will break it up. Recall that this sample setup has been taken from a live Cisco router that is loaded with 4 IPSEC policies. They are:

3DES/SHA/DH Group 5
3DES/MD5/DH Group 5
DES/SHA/DH Group 5
3DES/SHA/DH Group 2

Now also recall that in addition to the defined policy, the Linksys does:

3DES/SHA/DH Group 2
DES/MD5/DH Group 1
3DES/SHA/DH Group 2
3DES/MD5/DH Group 2.

Now the first part of the logs shows the linksys trying its 4 combinations against the 4 Cisco combinations. This takes a while.. Annotated comments inline

Feb  6 21:43:04.394 GMT: ISAKMP (0:0): received packet from 200.100.1.1 dport 500 sport 500 Global (N) NEW SA
Feb  6 21:43:04.394 GMT: ISAKMP: local port 500, remote port 500
Feb  6 21:43:04.394 GMT: ISAKMP: insert sa successfully sa = 64008FCC
Feb  6 21:43:04.394 GMT: ISAKMP (0:709): Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH

This is a MAIN MODE exchange. Not an aggressive mode exchange

Feb  6 21:43:04.394 GMT: ISAKMP (0:709): Old State = IKE_READY  New State = IKE_R_MM1
Feb  6 21:43:04.394 GMT: ISAKMP (0:709): processing SA payload. message ID = 0
Feb  6 21:43:04.398 GMT: ISAKMP: Looking for a matching key for 200.100.1.1 in default : success
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): found peer pre-shared key matching 200.100.1.1
Feb  6 21:43:04.398 GMT: ISAKMP (0:709) local preshared key found 

The preshared keys match and therefore the IKE peer is authenticated. Now we have to negotiate a common IKE SA policy between the peers to protect the IKE Exchange

Feb  6 21:43:04.398 GMT: ISAKMP : Scanning profiles for xauth …

Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Checking ISAKMP transform 1 against priority 1 policy
Feb  6 21:43:04.398 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.398 GMT: ISAKMP:      hash SHA
Feb  6 21:43:04.398 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.398 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.398 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.398 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Diffie-Hellman group offered does not match policy!

The cisco’s policy 1 is 3DES/SHA/DH Group 5 and this does not match Linksys polic 1 of DES/SHA/DH2

Feb  6 21:43:04.398 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Checking ISAKMP transform 2 against priority 1 policy
Feb  6 21:43:04.398 GMT: ISAKMP:      encryption DES-CBC
Feb  6 21:43:04.398 GMT: ISAKMP:      hash MD5
Feb  6 21:43:04.398 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.398 GMT: ISAKMP:      default group 1
Feb  6 21:43:04.398 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.398 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Encryption algorithm offered does not match policy!

The cisco’s policy 1 is 3DES/SHA/DH Group 5 and this does not match Linksys policy 2 of DES/MD5/DH2

Feb  6 21:43:04.398 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Checking ISAKMP transform 3 against priority 1 policy
Feb  6 21:43:04.398 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.398 GMT: ISAKMP:      hash SHA
Feb  6 21:43:04.398 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.398 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.398 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.398 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Diffie-Hellman group offered does not match policy!

Same as offer 1. The cisco’s policy 1 is 3DES/SHA/DH Group 5 and this does not match Linksys of 3DES/SHA/DH2

Feb  6 21:43:04.398 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Checking ISAKMP transform 4 against priority 1 policy
Feb  6 21:43:04.398 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.398 GMT: ISAKMP:      hash MD5
Feb  6 21:43:04.398 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.398 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.398 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.398 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Hash algorithm offered does not match policy!

The cisco’s policy 1 is 3DES/SHA/DH Group 5 and this does not match Linksys of 3DES/MD5/DH2

Feb  6 21:43:04.398 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 0
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Checking ISAKMP transform 1 against priority 2 policy
Feb  6 21:43:04.398 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.398 GMT: ISAKMP:      hash SHA
Feb  6 21:43:04.398 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.398 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.398 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.398 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Hash algorithm offered does not match policy!

The cisco’s policy 2 (note priority 2 policy above) is 3DES/MD5/DH Group 5 and this does not match Linksys of 3DES/SHA/DH2

Feb  6 21:43:04.398 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3

Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Checking ISAKMP transform 2 against priority 2 policy
Feb  6 21:43:04.398 GMT: ISAKMP:      encryption DES-CBC
Feb  6 21:43:04.398 GMT: ISAKMP:      hash MD5
Feb  6 21:43:04.398 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.398 GMT: ISAKMP:      default group 1
Feb  6 21:43:04.398 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.398 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Encryption algorithm offered does not match policy!The cisco’s policy 2 is 3DES/MD5/DH Group 5 and this does not match Linksys of DES/MD5/DH1Feb  6 21:43:04.398 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Checking ISAKMP transform 3 against priority 2 policy
Feb  6 21:43:04.398 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.398 GMT: ISAKMP:      hash SHA
Feb  6 21:43:04.398 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.398 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.398 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.398 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Hash algorithm offered does not match policy!The cisco’s policy 2 is 3DES/MD5/DH Group 5 and this does not match Linksys of 3DES/SHA/DH2Feb  6 21:43:04.398 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3
Feb  6 21:43:04.398 GMT: ISAKMP (0:709): Checking ISAKMP transform 4 against priority 2 policy
Feb  6 21:43:04.398 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.398 GMT: ISAKMP:      hash MD5
Feb  6 21:43:04.398 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.398 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.398 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.398 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Diffie-Hellman group offered does not match policy!The cisco’s policy 2 is 3DES/MD5/DH Group 5 and this does not match Linksys of 3DES/MD5/DH2Feb  6 21:43:04.402 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 0
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Checking ISAKMP transform 1 against priority 3 policy
Feb  6 21:43:04.402 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.402 GMT: ISAKMP:      hash SHA
Feb  6 21:43:04.402 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.402 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.402 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.402 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Encryption algorithm offered does not match policy!
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3The cisco’s policy 3 is DES/SHA/DH Group 5 and this does not match Linksys of 3DES/SHA/DH2

Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Checking ISAKMP transform 2 against priority 3 policy
Feb  6 21:43:04.402 GMT: ISAKMP:      encryption DES-CBC
Feb  6 21:43:04.402 GMT: ISAKMP:      hash MD5
Feb  6 21:43:04.402 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.402 GMT: ISAKMP:      default group 1
Feb  6 21:43:04.402 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.402 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Hash algorithm offered does not match policy!
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3

The cisco’s policy 3 is DES/SHA/DH Group 5 and this does not match Linksys of DES/MD5/DH1

Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Checking ISAKMP transform 3 against priority 3 policy
Feb  6 21:43:04.402 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.402 GMT: ISAKMP:      hash SHA
Feb  6 21:43:04.402 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.402 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.402 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.402 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Encryption algorithm offered does not match policy!
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 3

The cisco’s policy 3 is DES/SHA/DH Group 5 and this does not match Linksys of 3DES/SHA/DH2

Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Checking ISAKMP transform 4 against priority 3 policy
Feb  6 21:43:04.402 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.402 GMT: ISAKMP:      hash MD5
Feb  6 21:43:04.402 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.402 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.402 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.402 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Encryption algorithm offered does not match policy!
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): atts are not acceptable. Next payload is 0

The cisco’s policy 3 is DES/SHA/DH Group 5 and this does not match Linksys of 3DES/MD5/DH2

Feb  6 21:43:04.402 GMT: ISAKMP (0:709): Checking ISAKMP transform 1 against priority 4 policy
Feb  6 21:43:04.402 GMT: ISAKMP:      encryption 3DES-CBC
Feb  6 21:43:04.402 GMT: ISAKMP:      hash SHA
Feb  6 21:43:04.402 GMT: ISAKMP:      auth pre-share
Feb  6 21:43:04.402 GMT: ISAKMP:      default group 2
Feb  6 21:43:04.402 GMT: ISAKMP:      life type in seconds
Feb  6 21:43:04.402 GMT: ISAKMP:      life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:04.402 GMT: ISAKMP (0:709): atts are acceptable. Next payload is 3

Aha! We finally Hit policy 4 on the Cisco! 3DES/SHA/DH Group 2.

(This illustrates how advantageous it is to agree on a common IKE policy. Since each end has multiple combinations, there can be an increased number of combinations! Can be difficult though when dealing with vendor interoperability and 3DES export restrictions)

Now we deal with the second Main Mode Exchange. This is a DH exchange to generate shared secret keys.

You will see MMx which is Cisco’s state transitions as each sequence is performed.

MM1 – send an SA setup packet.

Feb  6 21:43:04.434 GMT: ISAKMP (0:709): Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE
Feb  6 21:43:04.438 GMT: ISAKMP (0:709): Old State = IKE_R_MM1  New State = IKE_R_MM1
Feb  6 21:43:04.438 GMT: ISAKMP (0:709): sending packet to 200.100.1.1 my_port 500 peer_port 500 (R) MM_SA_SETUP
Feb  6 21:43:04.438 GMT: ISAKMP (0:709): Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE

Packet send successfully therefore transition to MM2

Feb  6 21:43:04.438 GMT: ISAKMP (0:709): Old State = IKE_R_MM1  New State = IKE_R_MM2
Feb  6 21:43:05.378 GMT: ISAKMP (0:709): received packet from 200.100.1.1 dport 500 sport 500 Global (R) MM_SA_SETUP
Feb  6 21:43:05.378 GMT: ISAKMP (0:709): Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH

We received the IKE packet back from the peer. Now we move from MM2 to MM3

Feb  6 21:43:05.378 GMT: ISAKMP (0:709): Old State = IKE_R_MM2  New State = IKE_R_MM3
Feb  6 21:43:05.382 GMT: ISAKMP (0:709): processing KE payload. message ID = 0
Feb  6 21:43:05.422 GMT: ISAKMP (0:709): processing NONCE payload. message ID = 0
Feb  6 21:43:05.422 GMT: ISAKMP: Looking for a matching key for 200.100.1.1 in default : success
Feb  6 21:43:05.422 GMT: ISAKMP (0:709): found peer pre-shared key matching 200.100.1.1
Feb  6 21:43:05.422 GMT: ISAKMP (0:709): SKEYID state generated
Feb  6 21:43:05.422 GMT: ISAKMP (0:709): Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE
Feb  6 21:43:05.422 GMT: ISAKMP (0:709): Old State = IKE_R_MM3  New State = IKE_R_MM3
Feb  6 21:43:05.422 GMT: ISAKMP (0:709): sending packet to 200.100.1.1 my_port 500 peer_port 500 (R) MM_KEY_EXCH
Feb  6 21:43:05.422 GMT: ISAKMP (0:709): Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE

We received the DH key info from the peer and it checks out! Great. Now we do the thirsd MM exchange. We verify the peers identity uising IP address

Sending our packet out. Transition to MM4

Feb  6 21:43:05.422 GMT: ISAKMP (0:709): Old State = IKE_R_MM3  New State = IKE_R_MM4
Feb  6 21:43:06.942 GMT: ISAKMP (0:709): received packet from 200.100.1.1 dport 500 sport 500 Global (R) MM_KEY_EXCH
Feb  6 21:43:06.942 GMT: ISAKMP (0:709): Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH

Successfully received packet from peer. to MM5

Feb  6 21:43:06.942 GMT: ISAKMP (0:709): Old State = IKE_R_MM4  New State = IKE_R_MM5
Feb  6 21:43:06.942 GMT: ISAKMP (0:709): processing ID payload. message ID = 0
Feb  6 21:43:06.942 GMT: ISAKMP (0:709): ID payload
next-payload : 8
type         : 1
address      : 200.100.1.1
protocol     : 0
port         : 0
length       : 12

At this point, the peer (the Linksys) has sent its identity info as per the 3rd main mode exchange of IKE Phase 1. The error message below is a bit of a red herring. It relates to a specific IPSEC config the router checks for. See here for more info. http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/122newft/122t/122t15/ft_vrfip.htm
It can be safely ignored..

Feb  6 21:43:06.942 GMT: ISAKMP (0:709): peer matches *none* of the profiles

Now we need to check the hash of the identity data

Feb  6 21:43:06.942 GMT: ISAKMP (0:709): processing HASH payload. message ID = 0
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): SA authentication status:
Feb  6 21:43:06.946 GMT:  authenticated
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): SA has been authenticated with 200.100.1.1

Great – everything checks out. Again, ignore the next message

Feb  6 21:43:06.946 GMT: ISAKMP (0:709): peer matches *none* of the profiles

Now we send our ID info to the Linksys peer to complete the 3rd exchange of Main Mode Phase 1.

Feb  6 21:43:06.946 GMT: ISAKMP (0:709): Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): Old State = IKE_R_MM5  New State = IKE_R_MM5
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): SA is doing pre-shared key authentication using id type ID_IPV4_ADDR
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): ID payload
next-payload : 8
type         : 1
address      : 200.56.4.1
protocol     : 17
port         : 500
length       : 12
Feb  6 21:43:06.946 GMT: ISAKMP (709): Total payload length: 12
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): sending packet to 200.100.1.1 my_port 500 peer_port 500 (R) MM_KEY_EXCH
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE

Great! It looks like it all has worked as expected.

Feb  6 21:43:06.946 GMT: ISAKMP (0:709): Old State = IKE_R_MM5  New State = IKE_P1_COMPLETE
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): Input = IKE_MESG_INTERNAL, IKE_PHASE1_COMPLETE
Feb  6 21:43:06.946 GMT: ISAKMP (0:709): Old State = IKE_P1_COMPLETE  New State = IKE_P1_COMPLETE

Wohoo! Phase 1 is complete. Now we move to Phase 2 and do some real work. Now in Phase II, there is one mode called Quick mode and it is a 3 way exchange. First we get a packet from the originating peer – the Linksys. We check the hash and then the security association payload.

Feb  6 21:43:07.894 GMT: ISAKMP (0:709): received packet from 200.100.1.1 dport 500 sport 500 Global (R) QM_IDLE     
Feb  6 21:43:07.894 GMT: ISAKMP: set new node -841765728 to QM_IDLE     
Feb  6 21:43:07.894 GMT: ISAKMP (0:709): processing HASH payload. message ID = -841765728
Feb  6 21:43:07.894 GMT: ISAKMP (0:709): processing SA payload. message ID = -841765728

We have an IPSEC proposal to examine. Does it match our end?

Feb  6 21:43:07.894 GMT: ISAKMP (0:709): Checking IPSec proposal 1
Feb  6 21:43:07.894 GMT: ISAKMP: transform 1, ESP_3DES
Feb  6 21:43:07.894 GMT: ISAKMP:   attributes in transform:
Feb  6 21:43:07.894 GMT: ISAKMP:      SA life type in seconds
Feb  6 21:43:07.894 GMT: ISAKMP:      SA life duration (VPI) of  0×0 0×1 0×51 0×80
Feb  6 21:43:07.898 GMT: ISAKMP:      group is 2
Feb  6 21:43:07.898 GMT: ISAKMP:      encaps is 1 (Tunnel)
Feb  6 21:43:07.898 GMT: ISAKMP:      authenticator is HMAC-SHA

Yes this proposal matches! Now we send our proposal and acceptance!

Feb  6 21:43:07.898 GMT: ISAKMP (0:709): atts are acceptable.
Feb  6 21:43:07.898 GMT: IPSEC(validate_proposal_request): proposal part #1,
  (key eng. msg.) INBOUND local= 200.56.4.1, remote= 200.100.1.1,
    local_proxy= 192.168.100.0/255.255.255.0/0/0 (type=4),
    remote_proxy= 172.18.30.0/255.255.255.0/0/0 (type=4),
    protocol= ESP, transform= esp-3des esp-sha-hmac  (Tunnel),
    lifedur= 0s and 0kb,
    spi= 0×0(0), conn_id= 0, keysize= 0, flags= 0×22

Feb  6 21:43:07.930 GMT: ISAKMP (0:709): processing NONCE payload. message ID = -841765728
Feb  6 21:43:07.930 GMT: ISAKMP (0:709): processing KE payload. message ID = -841765728
Feb  6 21:43:07.974 GMT: ISAKMP (0:709): processing ID payload. message ID = -841765728
Feb  6 21:43:07.974 GMT: ISAKMP (0:709): processing ID payload. message ID = -841765728
Feb  6 21:43:07.974 GMT: ISAKMP (0:709): asking for 1 spis from ipsec
Feb  6 21:43:07.974 GMT: ISAKMP (0:709): Node -841765728, Input = IKE_MESG_FROM_PEER, IKE_QM_EXCH
Feb  6 21:43:07.974 GMT: ISAKMP (0:709): Old State = IKE_QM_READY  New State = IKE_QM_SPI_STARVE
Feb  6 21:43:07.974 GMT: IPSEC(key_engine): got a queue event…
Feb  6 21:43:07.974 GMT: IPSEC(spi_response): getting spi 1387857340 for SA
from 200.56.4.1   to 200.100.1.1  for prot 3
Feb  6 21:43:07.974 GMT: ISAKMP: received ke message (2/1)
Feb  6 21:43:08.226 GMT: ISAKMP (0:709): sending packet to 200.100.1.1 my_port 500 peer_port 500 (R) QM_IDLE

Okay we have sent our Quick Mode reply packet to the Linksys

Feb  6 21:43:08.226 GMT: ISAKMP (0:709): Node -841765728, Input = IKE_MESG_FROM_IPSEC, IKE_SPI_REPLY
Feb  6 21:43:08.226 GMT: ISAKMP (0:709): Old State = IKE_QM_SPI_STARVE  New State = IKE_QM_R_QM2

We have received the acknoweledgement from the peer and can proceed to build the IPSEC Security Association!

Feb  6 21:43:09.758 GMT: ISAKMP (0:709): received packet from 200.100.1.1 dport 500 sport 500 Global (R) QM_IDLE     
Feb  6 21:43:09.758 GMT: ISAKMP (0:709): Creating IPSec SAs
Feb  6 21:43:09.758 GMT:         inbound SA from 200.100.1.1 to 200.56.4.1 (f/i)  0/ 0
        (proxy 172.18.30.0 to 192.168.100.0)
Feb  6 21:43:09.762 GMT:         has spi 0×52B905BC and conn_id 5810 and flags 23
Feb  6 21:43:09.762 GMT:         lifetime of 86400 seconds
Feb  6 21:43:09.762 GMT:         has client flags 0×0
Feb  6 21:43:09.762 GMT:         outbound SA from 200.56.4.1   to 200.100.1.1  (f/i)  0/ 0 (proxy 192.168.100.0   to 172.18.30.0    )
Feb  6 21:43:09.762 GMT:         has spi -841765728 and conn_id 5811 and flags 2B
Feb  6 21:43:09.762 GMT:         lifetime of 86400 seconds
Feb  6 21:43:09.762 GMT:         has client flags 0×0
Feb  6 21:43:09.762 GMT: ISAKMP (0:709): deleting node -841765728 error FALSE reason "quick mode done (await)"
Feb  6 21:43:09.762 GMT: ISAKMP (0:709): Node -841765728, Input = IKE_MESG_FROM_PEER, IKE_QM_EXCH
Feb  6 21:43:09.762 GMT: ISAKMP (0:709): Old State = IKE_QM_R_QM2  New State = IKE_QM_PHASE2_COMPLETE

Yay! Phase II completed as well. Build that SA then!

Feb  6 21:43:09.762 GMT: IPSEC(key_engine): got a queue event…
Feb  6 21:43:09.762 GMT: IPSEC(initialize_sas): ,
  (key eng. msg.) INBOUND local= 200.56.4.1, remote= 200.100.1.1,
    local_proxy= 192.168.100.0/255.255.255.0/0/0 (type=4),
    remote_proxy= 172.18.30.0/255.255.255.0/0/0 (type=4),
    protocol= ESP, transform= esp-3des esp-sha-hmac  (Tunnel),
    lifedur= 86400s and 0kb,
    spi= 0×52B905BC(1387857340), conn_id= 5810, keysize= 0, flags= 0×23
Feb  6 21:43:09.762 GMT: IPSEC(initialize_sas): ,
  (key eng. msg.) OUTBOUND local= 200.56.4.1, remote= 200.100.1.1,
    local_proxy= 192.168.100.0/255.255.255.0/0/0 (type=4),
    remote_proxy= 172.18.30.0/255.255.255.0/0/0 (type=4),
    protocol= ESP, transform= esp-3des esp-sha-hmac  (Tunnel),
    lifedur= 86400s and 0kb,
    spi= 0xCDD3ACA0(3453201568), conn_id= 5811, keysize= 0, flags= 0×2B
 
Feb  6 21:43:09.762 GMT: IPSEC(crypto_ipsec_sa_find_ident_head): reconnecting with the same proxies and 200.100.1.1
Feb  6 21:43:09.762 GMT: IPSEC(add mtree): src 192.168.100.0, dest 172.18.30.0, dest_port 0Feb  6 21:43:09.762 GMT: IPSEC(create_sa): sa created,
  (sa) sa_dest= 200.56.4.1, sa_prot= 50,
    sa_spi= 0×52B905BC(1387857340),
    sa_trans= esp-3des esp-sha-hmac , sa_conn_id= 5810
Feb  6 21:43:09.762 GMT: IPSEC(create_sa): sa created,
  (sa) sa_dest= 200.100.1.1, sa_prot= 50,
    sa_spi= 0xCDD3ACA0(3453201568),
    sa_trans= esp-3des esp-sha-hmac , sa_conn_id= 5811
 
Done! Now lets use the Cisco show commands to have a look at the ISAKMP SA and IPSEC sa’s.

Checking IPSEC Status

RTCI01#sh cry
RTCI01#sh crypto isa
RTCI01#sh crypto isakmp sa
dst             src             state          conn-id slot
xxxxxx   200.100.1.1   QM_IDLE            673    0
200.56.4.1   200.100.1.1  QM_IDLE            709    0
xxxxxx   200.100.1.1   QM_IDLE            697    0
xxxxxx   200.100.1.1   QM_IDLE            674    0

RTCI01#sh crypto ipsec sa

interface: FastEthernet0/1
    Crypto map tag: IPSEC, local addr. 200.56.4.1

   protected vrf:
   local  ident (addr/mask/prot/port): (192.168.100.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (172.18.30.0/255.255.255.0/0/0)
   current_peer: 200.100.1.1:500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 0, #pkts encrypt: 0, #pkts digest 0
    #pkts decaps: 0, #pkts decrypt: 0, #pkts verify 0
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 200.56.4.1, remote crypto endpt.: 200.100.1.1
     path mtu 1500, media mtu 1500
     current outbound spi: CDD3ACA0

     inbound esp sas:
      spi: 0×52B905BC(1387857340)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        slot: 0, conn id: 5810, flow_id: 2827, crypto map: PIVoD
        sa timing: remaining key lifetime (k/sec): (4573743/3565)
        IV size: 8 bytes
        replay detection support: Y

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0xCDD3ACA0(3453201568)
        transform: esp-3des esp-sha-hmac ,
        in use settings ={Tunnel, }
        slot: 0, conn id: 5811, flow_id: 2828, crypto map: PIVoD
        sa timing: remaining key lifetime (k/sec): (4573743/3562)
        IV size: 8 bytes
        replay detection support: Y

     outbound ah sas:

     outbound pcp sas:

                      

No Tags