Welcome to the fourth post about the adaptive change that cloud computing is going to have on practitioners, paradigms and organisations. The previous two posts took a look at some of the dodgier side of two of the industries biggest players, Microsoft and Amazon. While I have highlighted some dumb issues with both, I nevertheless have to acknowledge their resourcing, scalability, and ability to execute. On that point of ability to execute, in this post we are going to expand a little towards the cloud industry more broadly and the inevitable consolidation that is, and will continue to take place.

Now to set the scene, a lot of people know that in the early twentieth century, there were a lot of US car manufacturers. I wonder if you can take a guess at the number of defunct car manufacturers there have been before and after that time.

…Fifty?

…One Hundred?

Not even close…

What if I told you that there were over 1700!

Here is another interesting stat. The table below shows the years where manufacturers went bankrupt or ceased operations. Below that I have put the average shelf life of each company for that decade.

Now, you would expect that the bulk of closures would be depression era, but note that the depression did not start until the late 1920’s and during the boom times that preceded it, 660 manufactures went to the wall – a worse result!

The pattern of consolidation

What I think the above table shows is the classic pattern of industry consolidation after an initial phase of innovation and expansion, where over time, the many are gobbled by the few. As the number of players consolidate, those who remain grow bigger, with more resources and economies of scale. This in turn creates barriers to entry for new participants. Accordingly, the rate of attrition slows down, but that is more due to the fact that there are fewer players in the industry. Those that are left continue to fight their battles, but now those battles take longer. Nevertheless, as time goes on, the number of players consolidate further.

If we applied a cloud/web hosting paradigm to the above table, I would equate the dotcom bust of 2000 with the depression era of the 1920’s and 1930’s. I actually think with cloud computing, we are in the 1960’s and on right now. The largest of the large players have how made big bets on the cloud and have entered the market in a big, big way. For more than a decade, other companies hosted Microsoft technology, with Microsoft showing little interest beyond selling licenses via them. Now Microsoft themselves are also the hosting provider. Does that mean most the hosting providers will have the fate of Netscape? Or will they manage to survive the dance with Goliath like Citrix or VMWare have?

For those who are not Microsoft or Amazon…

Imagine you have been hosting SharePoint solutions for a number of years. Depending on your size, you probably own racks or a cage in some-one else’s data centre, or you own a small data centre yourself. You have some high end VMWare gear to underpin your hosting offerings and you do both managed SharePoint (i.e. offer a basic site collection subscription with no custom stuff – ala Office 365) and you offer dedicated virtual machines for those who want more control (ala Amazon). You have dutifully paid your service provider licensing to Microsoft, have IT engineers on staff, some SharePoint specialists, a helpdesk and some dodgy sales guys – all standard stuff and life is good. You had a crack at implementing SharePoint multi tenancy, but found it all a bit too fiddly and complex.

Then Amazon comes along and shakes things up with their IaaS offerings. They are cost competitive, have more data centres in more regions, a higher capacity, more fault tolerance, a wider variety of services and can scale more than you can. Their ability to execute in terms of offering new services is impossible to keep up with. In short, they slowly but relentlessly take a chunk of the market and continue to grow. So, you naturally counter by pushing the legitimate line that you specialise in SharePoint, and as a result customers are in much more trusted hands than Amazon, when investing on such a complex tool as SharePoint.

But suddenly the game changes again. The very vendor who you provide cloud-based SharePoint services for, now bundles it with Exchange, Lync and offers Active Directory integration (yeah, yeah, I know there was BPOS but no-one actually heard of that). Suddenly the argument that you are a safer option than Amazon is shot down by the fact that Microsoft themselves now offer what you do. So whose hands are safer? The small hosting provider with limited resources or the multinational with billions of dollars in the bank who develops the product? Furthermore, given Microsoft’s advantage in being able to mobilise its knowledge resources with deep product knowledge, they have a richer managed service offering than you can offer (i.e. they offer multi tenancy .

This puts you in a bit of a bind as you are getting assailed at both ends. Amazon trumps you in the capabilities at the IaaS end and is encroaching in your space and Microsoft is assailing the SaaS end. How does a small fish survive in a pond with the big ones? In my opinion, the mid-tier SharePoint cloud providers will have to reinvent themselves.

The adaptive change…

So for the mid-tier SharePoint cloud provider grappling with the fact that their play area is reduced because of the big kids encroaching, there is only one option. They have to be really, really good in areas the big kids are not good at. In SharePoint terms, this means they have to go to places many don’t really want to go: they need to bolster their support offerings and move up the SharePoint stack.

You see, traditionally a SharePoint hosting provider tends to take two approaches. They provide a managed service where the customer cannot mess with it too much (i.e. Site collection admin access only). For those who need more than that, they will offer a virtual machine and wipe their hands of any maintenance or governance, beyond ensuring that  the infrastructure is fast and backed up. Until now, cloud providers could get away with this and the reason they take this approach should be obvious to anyone who has implemented SharePoint. If you don’t maintain operational governance controls, things can rapidly get out of hand. Who wants to deal with all that “people crap”? Besides, that’s a different skill set to typical skills required to run and maintain cloud services at the infrastructure layer.

So some cloud providers will kick and scream about this, and delude themselves into thinking that hosting and cloud services are their core business. For those who think this, I have news for you. The big boys think these are their core business too and they are going to do it better than you. This is now commodity stuff and a by-product of commoditisation is that many SharePoint consultancies are now cloud providers anyway! They sign up to Microsoft or Amazon and are able to provide a highly scalable SharePoint cloud service with all the value added services further up the SharePoint stack. In short, they combine their SharePoint expertise with Microsoft/Amazon’s scale.

Now on the issue of support, Amazon has no specific SharePoint skills and they never will. They are first and foremost a compelling IaaS offering. Microsoft’s support? … go and re-read part 2 if you want to see that. It seems that no matter the big multinational, level 1 tech support is always level 1 tech support.

So what strategies can a mid-tier provider take to stay competitive in this rapidly commoditising space. I think one is to go premium and go niche.

• Provide brilliant support. If I call you, day or night, I expect to speak to a SharePoint person straight away. I want to get to know them on a first name basis and I do not want to fight the defence mechanism of the support hierarchy.
• Partner with SharePoint consultancies or acquire consulting resources. The latter allows you to do some vertical integration yourself and broaden your market and offerings. A potential KPI for any SharePoint cloud provider should be that no support person ever says “sorry that’s outside the scope of what we offer.”
• Develop skills in the tools and systems that surround SharePoint or invest in SharePoint areas where skills are lacking. Examples include Project Server, PerformancePoint, integration with GIS, Records management and ERP systems. Not only will you develop competencies that few others have, but you can target particular vertical market segments who use these tools.
• (Controversial?) Dump your infrastructure and use Amazon in conjunction with another IaaS provider. You just can’t compete with their scale and price point. If you use them you will likely save costs, when combined with a second provider you can play the resiliency card and best of all … you can offer VPC

Conclusion

In the last two posts we looked at some of the areas where both Microsoft and Amazon sometimes struggle to come to grips with the SharePoint cloud paradigm. In this post, we took a look at other cloud providers having to come to grips with the SharePoint cloud paradigm of having to compete with these two giants, who are clearly looking to eke out as much value as they can from the cloud pie. Whether you agree with my suggested strategy (Rackspace appears to), the pattern of the auto industry serves as an interesting parallel to the cloud computing marketplace. Is the relentless consolidation a good thing? Probably not in the long term (we will tackle that issue in the last post in this series). In the next post, we are going to shift our focus away from the cloud providers themselves, and turn our gaze to the internal IT departments – who until now, have had it pretty good. As you will see, a big chunk of the irrational side of cloud computing comes from this area.

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

                      

No Tags

My next door neighbour is a builder. When he moved next door, the house was an old piece of crap. Within 6 months, he completely renovated it himself, adding in two bedrooms, an underground garage and all sorts of cool stuff. On the other hand, I bought my house because it was a good location, someone had already renovated it and all we had to do was move in. The reason for this was simple: I had a new baby and more importantly, me and power tools do not mix. I just don’t have the skills, nor the time to do what my neighbour did.

You can probably imagine what would happen if I tried to renovate my house the way my neighbour did. It would turn out like the Ikea fails in the video. Similarly, many SharePoint installs tend to look similar to the video too. Moral of the story? Sometimes it is better to get something pre-packaged than to do it yourself.

In the last post, we examined the “Software as a Service” (SaaS) model of cloud computing in the form of Office 365. Other popular SaaS providers include SlideShare, Salesforce, Basecamp and Tom’s Planner to name a few. Most SaaS applications are browser based and not as feature rich or complex as their on-premise competition. Therefore the SaaS model is that its a bit like buying a kit home. In SaaS, no user of these services ever touches the underlying cloud infrastructure used to provide the solution, nor do they have a full mandate to tweak and customise to their hearts content. SaaS is basically predicated on the notion that someone else will do a better set-up job than you and the old 80/20 rule about what features for an application are actually used.

Some people may regard the restrictions of SaaS as a good thing – particularly if they have dealt with the consequences of one too many unproductive customization efforts previously. As many SharePointer’s know, the more you customise SharePoint, the less resilient it gets. Thus restricting what sort of customisations can be done in many circumstances might be a wise thing to do.

Nevertheless, this actually goes against the genetic traits of pretty much every Australian male walking the planet. The reason is simple: no matter how much our skills are lacking or however inappropriate tools or training, we nevertheless always want to do it ourselves. This brings me onto our next cloud provider: Amazon, and their Infrastructure as a Service (IaaS) model of cloud based services. This is the ultimate DIY solution for those of us that find SaaS to cramping our style. Let’s take a closer look shall we?

Amazon in a nutshell

Okay, I have to admit that as an infrastructure guy, I am genetically predisposed to liking Amazon’s cloud offerings. Why? well as an infrastructure guy, I am like my neighbour who renovated his own house. I’d rather do it all myself because I have acquired the skills to do so. So for any server-hugging infrastructure people out there who are wondering what they have been missing out on? Read on… you might like what you see.

Now first up, its easy for new players to get a bit intimidated by Amazon’s bewildering array of offerings with brand names that make no sense to anybody but Amazon… EC2, VPC, S3, ECU, EBS, RDS, AMI’s, Availability Zones – sheesh! So I am going to ignore all of their confusing brand names and just hope that you have heard of virtual machines and will assume that you or your tech geeks know all about VMware. The simplest way to describe Amazon is VMWare on steroids. Amazon’s service essentially allows you to create Virtual Machines within Amazon’s “cloud” of large data centres around the world. As I stated earlier, the official cloud terminology that Amazon is traditionally associated is called Infrastructure as a Service (IaaS). This is where, instead of providing ready-made applications like SaaS, a cloud vendor provides lower level IT infrastructure for rent. This consists of stuff like virtualised servers, storage and networking.

Put simply, utilising Amazon, one can deploy virtual servers with my choice of operating system, applications, memory, CPU and disk configuration. Like any good “all you can eat” buffet, one is spoilt for choice. One simply chooses an Amazon Machine Image (AMI) to use as a base for creating a virtual server. You can choose one of Amazon’s pre-built AMI’s (Base installs of Windows Server or Linux) or you can choose an image from the community contributed list of over 8000 base images. Pretty much any vendor out there who sells a turn-key solution (such as those all-in-one virus scanning/security solutions) has likely created an AMI. Microsoft have also gotten in on the Amazon act and created AMI’s for you, optimised by their product teams. Want SQL 2008 the way Microsoft would install it? Choose the Microsoft Optimized Base SQL Server 2008R2 AMI  which “contains scripts to install and optimize SQL Server 2008R2 and accompanying services including SQL Server Analysis services, SQL Server Reporting services, and SQL Server Integration services to your environment based on Microsoft best practices.”

The series of screen shots below shows the basic idea. After signing up, use the “Request instance wizard” to create a new virtual server by choosing an AMI first. In the example below, I have shown the default Amazon AMI’s under “Quick start” as well as the community AMI’s.

Amazons default AMI’s

Community contributed AMI’s

From the list above, I have chosen Microsoft’s “Optimized SQL Server 2008 R2 SP1” from the community AMI’s and clicked “Select”. Now I can choose the CPU and memory configurations. Hmm how does a 16 core server sound with 60 gig of RAM? That ought to do it…

Now I won’t go through the full description of commissioning virtual servers, but suffice to say that you can choose which geographic location this server will reside within Amazon’s cloud and after 15 minutes or so, your virtual server will be ready to use. It can be assigned a public IP address, firewall restricted and then remotely managed as per any other server. This can all be done programmatically too. You can talk to Amazon via web services start, monitor, terminate, etc. as many virtual machines as you want, which allows you to scale your infrastructure on the fly and very quickly. There are no long procurement times and you then only pay for what servers are currently running. If you shut them down, you stop paying.

But what makes it cool…

Now I am sure that some of you might be thinking “big deal…any virtual machine hoster can do that.” I agree – and when I first saw this capability I just saw it as a larger scale VMWare/Xen type deployment. But when really made me sit up and take notice was Amazon’s Virtual Private Cloud (VPC) functionality. The super-duper short version of VPC is that it allows you extend your corporate network into the Amazon cloud. It does this by allowing you to define your own private network and connecting to it via site-to-site VPN technology. To describe how it works, diagrammatically check out the image below.

Let’s use an example to understand the basic idea. Let’s say your internal IP address range at your office is 10.10.10.0 to 10.10.10.255 (a /24 for the geeks). With VPC you tell Amazon “I’d like a new IP address range of 10.10.11.0 to 10.10.11.255” . You are then prompted to tell Amazon the public IP address of your internet router. The screenshots below shows what happens next:

The first screenshot asks you to choose what type of router is at your end. Available choices are Cisco, Juniper, Yamaha, Astaro and generic. The second screenshot shows you a sample configuration that is downloaded. Now any Cisco trained person reading this will recognise what is going on here. This is the automatically generated configuration to be added to an organisations edge router to create an IPSEC tunnel. In other words, we have extended our corporate network itself into the cloud. Any service can be run on such a network – not just SharePoint. For smaller organisations wanting the benefits of off-site redundancy without the costs of a separate datacenter, this is a very cost effective option indeed.

For the Cisco geeks, the actual configuration is two GRE tunnels that are IPSEC encrypted. BGP is used for route table exchange, so Amazon can learn what routes to tunnel back to your on-premise network. Furthermore Amazon allows you to manage firewall settings at the Amazon end too, so you have an additional layer of defence past your IPSEC router.

This is called Virtual Private Cloud (VPC) and when configured properly is very powerful. Note the “P” is for private. No server deployed to this subnet is internet accessible unless you choose it to be. This allows you to extend your internal network into the cloud and gain all the provisioning, redundancy and scalability benefits without exposure to the internet directly. As an example, I did a hosted SharePoint extranet where we use SQL log shipping of the extranet content databases back to the a DMZ network for redundancy. Try doing that on Office365!

This sort of functionality shows that Amazon is a mature, highly scalable and flexible IaaS offering. They have been in the business for a long time and it shows because their full suite of offerings is much more expansive than what I can possibly cover here. Accordingly my Amazon experiences will be the subject of a more in-depth blog post or two in future. But for now I will force myself to stop so the non-technical readers don’t get too bored.

So what went wrong?

So after telling you how impressive Amazon’s offering is, what could possibly go wrong? Like the Office365 issue covered in part 2, absolutely nothing with the technology. To understand why, I need to explain Amazon’s pricing model.

Amazon offer a couple of ways to pay for servers (called instances in Amazon speak). An on-demand instance is calculated based on a per-hour price while the server is running. The more powerful the server is in terms of CPU, memory and disk, the more you pay. To give you an idea, Amazon’s pricing for a Windows box with 8CPU’s and 16GB of RAM, running in Amazon’s “US east” region will set you back $0.96 per hour (as of 27/12/11).  If you do the basic math for that, it equates to around $8409 per year, or $25228 over three years. (Yeah I agree that’s high – even when you consider that you get all the trappings of a highly scalable and fault tolerant datacentre).

On the other hand, a reserved instance involves making a one-time payment and in turn, receive a significant discount on the hourly charge for that instance. Essentially if you are going to run an Amazon server on a 24*7 basis for more than 18 months or so, a reserved instance makes sense as it reduces considerable cost over the long term. The same server would only cost you $0.40 per hour if you pay an up-front $2800 for a 3 year term. Total cost: $13312 over three years – much better.

So with that scene set, consider this scenario: Back at the start of 2011, a client of mine consolidated all of their SharePoint cloud services to Amazon from a variety of other another hosting providers. They did this for a number of reasons, but it basically boiled down to the fact they had 1) outgrown the SaaS model and 2) had a growing number of clients. As a result, requirements from clients were getting more complicated and beyond that which most of the hosting providers could cater for. They also received irregular and inconsistent support from their existing providers, as well as some unexpected downtime that reduced confidence. In short, they needed to consolidate their cloud offering and manage their own servers. They were developing custom SharePoint solutions, needed to support federated claims authentication and required disaster recovery assurance to mitigate the risk of going 100% cloud. Amazon’s VPC offering in particular seemed ideal, because it allowed full control of the servers in a secure way.

Now making this change was not something we undertook lightly. We spent considerable time researching Amazon’s offerings, trying to understand all the acronyms as well as their fine print. (For what its worth I used IBIS as the basis to develop an assessment and the map of my notes can be found here). As you are about to see though, we did not check well enough.

Back when we initially evaluated the VPC offering, it was only available in very few Amazon sites (two locations in the USA only) and the service was still in beta. This caused us a bit of a dilemma at the time because of the risk of relying on a beta service. But we were assured when Amazon confirmed that VPC would eventually be available in all of of their datacentres. We also stress tested the service for a few weeks, it remained stable and we developed and tested a disaster recovery strategy involving SQL log shipping and a standby farm. We also purchased reserved instances from Amazon since these servers were going to be there for the long haul, so we pre-paid to reduce the hourly rates. Quite a complex configuration was provisioned in only two days and we were amazed by how easy it all was.

Things hummed along for 9 months in this fashion and the world was a happy place. We were delighted when Amazon notified us that VPC had come out of beta and was now available in any of Amazon’s datacentres around the world. We only used the US datacentre because it was the only location available at the time. Now we wanted to transfer the services to Singapore. My client contacted Amazon about some finer points on such a move and was informed that they would have to pay for their reserved instances all over again!

What the?

It turns out, reserved instances are not transferrable! Essentially, Amazon were telling us that although we paid for a three year reserved instance, and only used it for 9 months, to move the servers to a new region would mean we have to pay all over again for another 3 year reserve. According to Amazon’s documentation, each reserved instance is associated with a specific region, which is fixed for the lifetime of the reserved instance and cannot be changed.

“Okay,” we answer, “we can understand that in circumstances where people move to another cloud provider. But in our case we were not.” We had used around 1/3rd of the reserved instance. So surely Amazon should pro-rata the unused amount, and offer that as a credit when we re-purchase reserved instances in Singapore? I mean, we will still be hosting with Amazon, so overall, they will not be losing any revenue al all. On the contrary, we will be paying them more, because we will have to sign up for an additional 3 years of reserve when we move the services.

So we ask Amazon whether that can be done. “Nope," comes back the answer from amazons not so friendly billing team with one of those trite and grossly insulting “Sorry for any inconvenience this causes” ending sentences. After more discussions, it seems that internally within Amazon, each region or datacentre within each region is its own profit centre. Therefore in typical silo fashion, the US datacentre does not want to pay money to the Singapore operation as that would mean the revenue we paid would no longer recognised against them.

Result? Customer is screwed all because the Amazon fiefdoms don’t like sharing the contents of the till. But hey – the regional managers get their bonuses right?

Conclusion

Like part 2 of this cloud computing series, this is not a technical issue. Amazon’s cloud service in our experience has been reliable and performed well. In this case, we are turned off by the fact that their internal accounting procedures create a situation that is not great for customers who wish to remain loyal to them. In a post about the danger of short termism and ignoring legacy, I gave the example of how dumb it is for organisations to think they are measuring success based on how long it takes to close a helpdesk call. When such a KPI is used, those in support roles have little choice but to try and artificially close calls when users problems have not been solved because that’s how they are deemed to be performing well. The reality though is rather than measure happy customers, this KPI simply rewards which helpdesk operators have managed to game the system by getting callers off the phone as soon as they can.

I feel that Amazon are treating this is an internal accounting issue, irrespective of client outcomes. Amazon will lose the business of my client because of this since they have enough servers hosted where the financial impost of paying all over again is much more than transferring to a different cloud provider. While VPC and automated provisioning of virtual servers is cool and all, at the end of the day many hosting providers can offer this if you ask them. Although it might not be as slick with fancy as Amazon’s automated configuration, it nonetheless is very doable and the other providers are playing catch-up. Like Apple, Amazon are enjoying the benefits of being first to market with their service, but as competition heats up, others will rapidly bridge the gap.

Thanks for reading

Paul Culmsee

                      

No Tags

This third post in the “Who do SharePoint Projects Fail” series has been hard to write, not because I am struggling with ideas, but because I have too many! It is hard to put all the reasons why SharePoint projects go wrong into a coherent chain of logic.

In the first two posts in this series, we did a basic examination of wicked problem theory.

Part 1 introduced you to tequila slammers, as well as the pioneering work by Horst Rittel and the concept of wicked problems.

Part 2 also delved into the murky depths of academic history to demonstrate that even back in the seventies when ABBA stole the hearts and minds of teenyboppers around the world, at least some people had time to look at wicked problems in relation to building IT systems.

If you take away anything from part 1 and 2, it is this.

• Too many tequila slammers hurt
• Before you blame the product, the project manager, the stakeholders, the nerds, the methodology or anything else in vicinity, go back to the problem you are solving and determine its ‘wickedness’

Now we will finally look at this large, complex, scary beast known as SharePoint. I have no means to quantify how much of a percentage of project problems arise from issues related to “the product”, but it definitely happens. Unsurprisingly enough, it is easy to argue that some of the areas that I highlight below are people issues, but we still get to indulge in Microsoft bashing – and who doesn’t enjoy a bit of that eh?

The Microsoft Effect – Good enough…

Microsoft in some ways remind me of Brittany Spears*. Both crash things, both can be abusive, both get themselves into trouble with the law, both have ignored legal judgements, both have loyal fans who love them no matter what and detractors that will never accept they can do anything right, both get caught with their pants down on occasion, and both have produced some pretty ordinary crap in their time. Despite this, they sell squillions of copies and make a ton of money.

*Just in case you read this article at some future point where Brittany is long forgotten, substitute whichever starlet is in rehab

But Microsoft is *big*, Microsoft are *safe*. We already use Microsoft in the organisation now and thus Microsoft products *integrate well*.

So what is it about SharePoint that makes it such a magnet for a wicked project? Put quite simply, with SharePoint they have taken integration to another level. Microsoft have done their homework and created a product/platform that has a combination of the right kind of capabilities that *together* can be extremely compelling. Each component, judged individually as a point solution would not necessarily compete against dedicated products in the space. But Microsoft long ago learned that a combination of tight integration and the old 80/20 rule usually wins.

There have been document management systems and web content management for years. I’ve previously been involved with a large Hummingbird deployment. As soon as I saw MOSS 2007 I know it was going to massive, not because it was as good as Hummingbird, because it wasn’t. But it was *good enough* and it was made by Microsoft. Market success therefore was assured.

Easy to Impress – A Mythical Example

To demonstrate how effective this combination of features can be as a sales tool, consider this mythical conversation between a somewhat green, yet enthusiastic Microsoft evangelist named Bill Gates, who is discussing MOSS with key stakeholders at the company GOOSUNACLE. Their names are Scott McNealy, Sergei Brin, Steve Jobs and Larry Ellison.

(1 point for each in-joke you spot)

Scott McNealy (Infrastructure Manager): Okay Bill, here’s our key problem. We can’t find anything and our knowledge sharing is abysmal. We waste so much time trying to work out where stuff should be, and nobody follows the standards. People reinvent the wheel constantly and we keep making the same mistakes over and over again. Our file system really sucks with so much duplication and complexity. It has so much history there, that nobody wants to own it or fix it up. This has cost us market share and our bottom line is suffering. What can you do to help?

Bill Gates: Well Scotty, (do you mind if I call you Scotty?), I am glad you asked. SharePoint represents a new way to manage your enterprise documents. It can help you with built in version control, check-in/out, metadata classification and a recycle bin. Additionally, it is web based, making it much more accessible to your staff. It is also tightly integrated with Microsoft Office, particularly the 2007 version. We also have document format conversion features and records management too.

Sergei Brin (Knowledge Manager) : “Sweet!. Kind of makes the whole notion of a separate intranet and file system start to look ‘old school’. What about search? I feel very strongly that a good search engine is worth its weight in gold.

Bill Gates: That’s a great point you make Sergei, but I think it is best you leave search to us experts at Microsoft. Now in relation to your question, search is vitally important and SharePoint has a very powerful, scalable, customisable search engine that allows you to index all sorts of data sources. 

Larry Ellison (Database Administrator). Okay, so you can classify and search your files and other content. But I’m a database kind of guy. What about our line of business data in our databases? Can we improve its visibility to a larger audience?

Bill Gates: You certainly can Larry. We have several features that facilitate this, but specifically, we have the business data catalogue feature, which allows you to connect to 3rd party data sources for search, business intelligence and reporting. That way, you do not need to train your users in your line of business applications. They can simply visit the SharePoint portal, and view their dashboards and key performance indicators, so gain visibility into the business. In addition we integrate with SQL Reporting Services and can easily surface enterprise reports into the portal. Excel services is another feature that falls into this area. Imagine being able to publish your complex Excel sheets to the portal and users can interact with then without requiring excel on their desktops.

Scott McNealy: That is all very interesting indeed, document management, business intelligence and reporting in the one product. The other big issue we have is ensuring that people do things consistently. Larry and Steve in particular, keep releasing marketing slogans without formal approval. The “unbreakable” and “sooper dooper secure”  campaigns kind of backfired on us, and it would have been much better we had a more rigorous review process.

Bill Gates: Ah yes Scotty my lad, that is an excellent example. SharePoint contains a powerful workflow engine under the hood, that allows us to automate business process. So for example, at Microsoft when staff members leave the company, as part of our exit process we automatically create a task for the ever popular “throw the chair sendoff”. Supporting the workflow, is the document management features that I described earlier, as well as business forms via infopath and forms server.

Steve Jobs (Communications and Branding Manager). What about the GUI? It has to look good for people to engage with it. Additionally, I’d like the content to be available via mobile devices too.

Bill Gates: SharePoint has a powerful web content management foundation as well. It is built upon ASP.NET, which uses a templating system known as master pages, page layouts and web parts. But rather than me get into the detail here, I’ll recommend you visit my all time favourite blog called CleverWorkArounds and read the branding series. It should answer all of your questions. In relation to mobile devices, content can be displayed on these devices with no additional conversion. Suffice to say that it takes advantage of some of the workflow and document management features to facilitate content editorial approval, scheduled publication, and the like.

Steve Jobs: We are sold, Sergei, pass me the chequebook.

The downsides of versatility

So, Bill has sold Steve, Larry, Sergei and Scott on the possibilities of SharePoint. All of their respective buttons have been pushed and that can see it solving particular problems in the business. At a high level, they can see the value of re-use and efficiencies gained from using the same platform.

But which problem are we solving? If you asked them individually at this point, they would give you different answers. Its clear that Steve likes the WCM features, while Larry digs Business Intelligence. Sergey loves search and Scott wants to sort out document management.

Already, there is a wicked problem in the making, because it is easy to fall into the trap of “a tool looking for a problem” and nobody yet has the same definition of the problem. (one of Rittel’s key wicked problem warning signs that we covered in part 1)

The panacea effect

With so many features in SharePoint, combined in just the right way, it is the panacea you have been looking for. So you have found your cure, lets use it for everything right? The panacea solution often costs more, or is overkill for an initial problem, so to justify it requires you to sell it’s potential to solve other, bigger problems.

If you have read the first two articles in this series, you will realise pretty quickly that this is very high risk approach and can get out of hand pretty quickly. Just because a tool is versatile and has the potential to solve multiple problems does not mean it will necessarily be the best option to solve your particular problem.

Add the Microsoft effect to the panacea effect and the combination is more lethal! 

IT departments are particularly guilty of going down the panacea road and the reason this happens is also unsurprising when you think about it. IT are conditioned to think in terms of ‘tools and services’. By that I mean that, they exist to support and manage the organisations IT services. All business divisions have a desire to consolidate, streamline and reduce wastage and the IT department is no exception. In terms of an IT Department perspective, consolidating the number of “IT services” is attractive as there is tangible cost in managing many disparate systems.

So on the surface of it, SharePoint brings the IT department the potential tangible advantage of a common platform to support and manage.

I won’t belabour this point any further, because I have written about this issue on a couple of occasions, the best example being my choose your own adventure post and more specifically, its follow up. To the left is the book cover I did for the choose your own adventure. Sometimes pictures speak a thousand words.

It is also not fair to pick on IT either. I’ve gone out to many sites where the ‘SharePoint champion’ is not an IT person at all. But there is a consistent theme there. They want to use it because they have seen the potential for it to solve problems. Seeing the potential to solve a problem is not a problem statement. Therefore, by deploying this tool into the organisation, what problem are you solving and how do you determine if it has been effectively solved?

As a result of the panacea effect, we end up at another great debate of IT projects.

Point solution vs platform solution

I believe that as far as web content management products go, there are better point solutions around than the features SharePoint supply. WordPress for me is a much better blog system than SharePoint, Drupal as a pure WCM offers similar features to SharePoint for a fraction of the cost. Documentum is a better records management system by far and Cognos is going to trump SharePoint in the business intelligence stakes. Closer to home, in my SharePoint for Cisco nerds series, I had a comment that suggested that RANCID was a much better choice than SharePoint for Cisco configuration backups. He is absolutely right too, it is much better. But he is equally wrong too, depending on what the problem is and the point of view of the individual. (to find out why you have to read the whole series and judge for yourself)

This debate is essentially a version of the “Off the Shelf” vs “Custom Designed Software” chestnut. A point solution will meet your core requirements, however it may require you to change your business processes to work around it. Custom designed solutions can integrate nicely with your business process as they are to your exact specification. But the time and cost to develop can be significantly more and requirements can be fluid or prone to creep. (and you know how project managers hate that!)

Ah, but of course, being a point solution, the advantage of those products are diluted as your requirements broaden. You need multiple point solutions, those point solutions need to play nice with each other. The integration and support costs start to increase.

Although I see SharePoint as a platform solution in general, it actually has the best and worst elements of both off the shelf and custom designed software. It can cost you a lot of dollars and it all depends on your problem being solved. For example, it can be used purely as an application development platform, where a custom solution can take advantage of SharePoint features to underpin their own application, saving development time as you do not have to write certain functionality yourself. In this scenario it was always going to be a custom development job anyway, and SharePoint actually saves you money.

However, some problems are best fixed with a point solution and some are not. Where you can get into trouble is investing a lot of time and effort into a platform, because of the adherence to the panacea effect. The goal behind the platform may well be legitimate and well intentioned, but at the end of day, you are potentially talking a high development cost to get it exactly the way you want when a point solution may be a fraction of the cost.

I previously blogged a 6 part series on SharePoint ROI considerations, and although I haven’t blogged about it yet, there are financial modelling techniques available to consider some of the esoteric considerations like ‘opportunity cost’ of point vs platform. I will blog on this topic in the future, but for now, if you take anything away from this section, it is not to underestimate the value of the point solution when looking at your problem being solved.

Conclusion

*pauses for a big sigh*

That was a tough article to write, and the next one is probably going to be tougher, as I will no doubt have an urge to go back and edit this article. But I’ll resist that temptation as otherwise I’d never get these articles out.

But I hope that what you take away from this post is a deeper understanding of some of the reasons why a SharePoint project in particular can get out of hand and become wicked in nature. The quicker you spot them, the quicker you can head off the potential pain and suffering that will follow.

In the next post we will keep digging around SharePoint’s dirty little secrets and then look into some techniques that we can use to try and mitigate some of these issues.

cheers

Paul

                      

No Tags

Hey there!

Sorry it has taken me a while to get back to the Cisco articles. The “choose your own adventure” post took longer than I thought it would and I also was side tracked blogging about annoying programming issues with XML and Javascript.

This is the last of the Cisco fanboy series of articles and really its more a tidying up of loose ends. To call this last article a Cisco fanboy article is a bit of a stretch actually, since we are now moving to a broader level of governance and accountability, and is therefore not really about Cisco, so I’ll start a new series more appropriately titled and continue from where this article leaves off. 

I started the series with the intent of starting with a seemingly innocuous scenario (Cisco TFTP backups), demonstrating how SharePoint can be leveraged as an okay point solution. I then tried to slowly expand the scope to the broader issues of complex infrastructure management management, while sticking to a Cisco/IP network oriented theme in an attempt to get technical thinkers (like Cisco guys) to think beyond nuts and bolts. This also demonstrated how thinking past ‘the point solution’ can being more substantive benefits. 

Here is the quick recap on what we have covered so far:

• Part 1 illustrated how it it possible to use SNMP to tell a Cisco IOS device to dump its configuration to TFTP. We talked about the version control feature of SharePoint and why it makes sense to TFTP your configs to a SharePoint document library. We covered the WEBDAV network provider supplied with XP and Win2003 and finished off with a basic example using the TFTP server TFTPD32.
• Part 2 went into more detail about the issues you can face when using the WEBDAV network provider. It also went into more detail on 3 TFTP server products (WinAgents TFTP, SolarWinds TFTP Server and TFTPD32 and why TFTPD32 ended up being the best choice for this purpose.
• Part 3 then looked at a wonderful open source TFTP server written in C# called TFTPUtil. We modified the source code of this TFTP server to use the SharePoint SDK and upload files to a SharePoint document library.
• Part 4 continued the examination of the TFTPUtil server and another modification of it to use SharePoint web services to upload files. We then summarised the merits of each of the methods.
• Part 5 switched tack and introduced a real world applicable scenario of combining Cisco configuration backups along with diagrams, documentation, asset information and site/support information into a one-stop-shop integrated portal, utilising lists, libraries and workflow to integrate it all neatly.

As I said in my introduction, I intend to get into some more high level strategic stuff in relation to IT operations/infrastructure management in a forthcoming series of articles. So in this last set of articles, I will cover off one major remaining question that I think my technically minded readers would want to see addressed.

“But what if SharePoint dies”?

This is a very common query/issue I get. (Luckily, I have an answer that has satisfied all so far … touch wood  )

Although I have been able to convince even some of the most fervent anti-Micro$oft, anti-portal people the power that SharePoint can offer for knowledge management and collaboration, people quite rightly get a little nervous about the notion of all of your critical information sitting in a single application like SharePoint.

Certainly, SharePoint  is a damn complex suite of applications, meaning that it’s own backup and recovery is more complex than most applications. It requires forward planning, user and IT staff training, lots of governance, a relatively process mature IT department and a heck of a lot of other effort. “and after all of that you still want me to store all of our critical information there?”

I believe absolutely that it is well worth this effort, and a cheap tool released by the guys at Colligo make the whole question somewhat redundant with the release of their cheap (but excellent) “Colligo Reader”.

Colligo Reader is free for personal use and cheap for corporate use. It allows a one way replication of SharePoint portals for offline access. “Yeah but Outlook and Groove can do this”, I hear you say. but this product synchronises libraries, lists, forms, all the while preserving their views. Sweet!

Imagine the possibilities here. Your network administrators could run a continual sync of your Cisco infrastructure portal on their laptops. In the event of a major catastrophic fault where all IT services are unavailable (including SharePoint), you have an the most up to date copy of your documentation, support information and most recent network configuration.

Colligo Demonstrated

So let’s install the product and synchronise the http://tidemo/tftp site that we used in the previous articles to illustrate how useful Colligo is.

Starting Colligo reader for the first time presents you with a basic GUI as shown below.

The first thing to do is to choose a SharePoint site to synchronise. Click the site drop down and choose to download a site. Enter the URL of the SharePoint site to download.

Colligo Reader will now perform an initial sync of the site. Depending on how much content is stored on the site, this may take some time.

One the initial sync has completed, you see the main Colligo Reader screen, except this time we can see the site listed and the documents and lists that have been synchronised.

Now let’s examine some of the lists and document libraries created in part 5, that have been synchronised.

IT Asset Documentation

Note that this looks somewhat like a SharePoint document library. We have our views on the top right, and we can also see the documents and the metadata columns that were created for this library. (“Related Device” and “Document Type”). Below is the same document library, illustrating the “By Document Type” view.

For reference, here is the original SharePoint view to compare.

“Client Sites” List

Below is the synchronised “Client Sites” list. Like the document library example above, all columns and views are fully synchronised.

Caveats

There are only three issues that spring to mind when using Colligo Reader. All three could be addressed by Colligo is they felt so inclined.

The first caveat is that only the latest version of library/list data is replicated. So those 291 versions of myfile.txt I had in my http://tidemo/tftp site (see below ), only the latest one is replicated to Colligo.

I don’t see this as a major issue, as the whole point here is disaster recovery for when SharePoint is not available.

The second caveat is when a site is replicated, sub-sites are not replicated unless you explicitly add them to be downloaded. It would be great if you could tick a box that says to replicate the current site and all sub-sites.

The final caveat is the major one. I really wish that Colligo offered to password protect synchronised sites (on a site by site basis). While it is great to be able to have an up to date, offline copy of your critical infrastructure information, but some will then point to the obvious security implications of laptops containing this goldmine of sensitive information all nicely laid out for unauthorised parties to access.

My technical answer to this is to look at Vista/bitlocker (or 3rd party equivalent) encryption at an operating system level, since Colligo content isn’t necessarily the only sensitive content on a notebook or PC.

My non technical answer is that this is a typical risk vs reward scenario and the applicability of this solution is really dependent on your security posture.

Conclusion

Colligo Reader is an exceptionally useful tool for the offline synchronisation of certain types of SharePoint portals, in particular IT infrastructure and managed services portals make excellent candidates. Our example above shows that we now have VPN details, customer contacts, documentation, configuration backups and support contract details all readily available without requiring SharePoint being available.

So, when step back and look at all we have covered here (particularly the last two articles), we have managed to kick a few significant goals. What makes this really interesting is that we are using tools and techniques that is applicable not just for Cisco gear, but pretty much any other IT infrastructure, whether it is line of business applications, complex hardware or a combination of both.

Whatsmore, while not replacing the need for and value of a Business Impact Analysis and Disaster Recovery Plan, you have still in a significantly improved position if something bad was to occur.

As I said earlier, I’ll be revisiting this again, now more broadly, looking at this topic from the point of view of an IT Operations Manager who has compliance requirements. They are not interested in Cisco per se, but about the whole enchilada!

until then…

regards

Paul

                      

No Tags

Greetings SharePoint fans and Cisco people who are soon to be SharePoint fans! I’m back with part 5 of this series on leveraging features of MOSS 2007 and WSS 3.0 to help manage Cisco infrastructure.

For those of you who hate programming topics and have no interest in TFTP issues, we may well have finally gotten to a topic that you are interested in! For the rest of you, who have read the first four articles, we now shift our focus from getting Cisco configurations into SharePoint, to doing something useful with them once we have done so.

In this post, we will examine the out of the box capabilities of workflow in SharePoint and delve a little deeper into document libraries and content management. So this is more aimed at Cisco people than it is at SharePoint Pros. SharePoint people, much of the topics I cover in this article you may have seen before.

I personally don’t think this post has a high coffee requirement rating in terms of complexity of concepts, compared to the last two anyhow. However be prepared: it is a long read.

CleverWorkArounds Coffee requirement rating:

Here is the quick recap on what we have covered so far:

• Part 1 illustrated how it it possible to use SNMP to tell a Cisco IOS device to dump its configuration to TFTP. We talked about the version control feature of SharePoint and why it makes sense to TFTP your configs to a SharePoint document library. We covered the WEBDAV network provider supplied with XP and Win2003 and finished off with a basic example using the TFTP server TFTPD32.
• Part 2 went into more detail about the issues you can face when using the WEBDAV network provider. It also went into more detail on 3 TFTP server products (WinAgents TFTP, SolarWinds TFTP Server and TFTPD32 and why TFTPD32 ended up being the best choice for this purpose.
• Part 3 then looked at a wonderful open source TFTP server written in C# called TFTPUtil. We modified the source code of this TFTP server to use the SharePoint SDK and upload files to a SharePoint document library.
• Part 4 continued the examination of the TFTPUtil server and another modification of it to use SharePoint web services to upload files.

I think the best way to get stuck into part 5 is via a real-world Cisco scenario. For this scenario, I will (loosely) use one of my roles from a few years back. It was not the biggest or most complex network I’ve managed, but it was quite varied due to the nature of the business they were in.

The scenario

Our scenario company is called CleverWorkArounds Corp and it is a products and services company. This business is involved in digital signage primarily (you know all those LCD screens spamming you with information and advertisements in shopping malls, train stations, cinemas, fancy hotel lobbies, casinos, etc).

Below is an example of large scale digital signage. Look closely…is that a windows error I see?

There are many installations of the product all over the world. Some locations include countries with encryption restrictions or government firewalls. Many of the installations are contracted to be monitored in real-time and in many installs the day to day management is provided by CleverWorkArounds Corp as a managed service.

Including head office in the USA, CleverWorkArounds Corp has 5 offices around the world where the installations are put together and tested before being shipped to site.

The Network

The network infrastructure employed by CleverWorkArounds Corp is Cisco based. The most complex part about it is the VPN connectivity to office and client sites. There is a lot of variation. “Managed Service” type installations typically use CleverWorkArounds Corp supplied network gear on site, in a segmented or isolated network. However many installations are hosted on the client’s infrastructure, often not Cisco, and all CleverWorkArounds Corp has is a VPN connection to the site.

CleverWorkArounds Corp has implemented VLAN’s, wireless and multicast networking at many sites. The security set-up and configuration varies from site to site, as it depends on who owns the infrastructure, security posture of the client and the level of local support on the ground at that site.

Finally, even the VPN technology varies. Most sites are point-to-point IPSEC, but some require CleverWorkArounds Corp to use PPTP or SSL VPN via OpenVPN when firewalls/government restrictions get in the way.

The Issues

The most critical issue with this type of network is security. Not only does CleverWorkArounds Corp have to protect itself from it’s clients, since many client sites are VPN connected with varying levels of security posture, CleverWorkArounds Corp also has to protect client’s from each-other. A relatively complex set of access lists are maintained and authentication to all CleverWorkArounds Corp network devices is via RADIUS and central authentication.

Aside from security, the CleverWorkArounds Corp corporate offices are connected to each-other via GRE tunnels over IPSEC, which allows multicast traffic and as a side effect, dynamic routing protocols. EIGRP is the protocol used. Many ‘lab’ VLAN’s are running at any given time, while new installations are being put together.

On the VPN side, on top of the variation in underlying technology, some Network Address translation is performed where IP address ranges overlap, as many clients like to use the 10.0.0.0 network with a /8 subnet mask (255.0.0.0) and don’t realise how restrictive this can be when doing b2b connectivity.

So What Are We Interested In Here?

From the point of view of the business, all of this has to be up and running, humming along with no service disruption as downtime = lost $. No doubt you have taken out SMARTnet maintenance contracts right? .

So you need to have all the tools and critical information on hand to recover as quickly as possible.

In my career, I’ve been dumped into the deep end a few times, inheriting pathetically maintained/managed infrastructure that is undocumented and a ticking time-bomb. Is this picture familiar?

Thus I always like to leave a site or infrastructure well documented. However my philosophy here is not to rewrite the manual. I do make some assumptions, including the assumption that the next person taking over at least has some semblance of previous experience and understanding of the products making up the infrastructure.

Therefore I do not attempt to rewrite product manuals. I will write a general network SOE document to go with a layer 2 and layer 3 network diagram. The SOE will cover implementation details of the network design including RADIUS, SNMP, EIGRP, Spanning tree, VPN, VLAN’s, IP Telephony, etc. But the SOE will not explain what each of these technologies are and what they do – that is assumed information.

That document is around 40-60 pages usually and once written, it is fairly easy to maintain because it is not filled with constantly changing operation information. That is kept separately. For example I’ll maintain lists of each client site, including the IP range used, details and serial numbers of devices on site, maintenance agreement numbers, client contact details, passwords, VPN details such as the certificate or shared key. I’ll document what each VLAN is for, SNMP community string, etc.

Finally, in case you did not know already , I implement my perl script to back up the configurations of all of the IOS devices to TFTP. (And that perl script is well documented in the network SOE I described earlier).

So this is all very nice and standard practice, and we can do this without SharePoint just fine, but let’s go through an simple example of maintaining this information in SharePoint and see the benefits.

SharePoint Revisited

First up, let’s now modify the SharePoint site (http://tidemo/tftp) we were using for the TFTP experimentation. We will add some additional columns and lists. This will allow us to track much more interesting information than just the backups of router configs.

CleverWorkArounds caveat: This is an example and does not reflect reality! It purely serves to illustrate SharePoint’s capabilities. In addition, this is not meant as a complete newbie SharePoint tutorial or this post would get far too big – I have skipped some details here.

“Clients” list

First up, we have a number of clients and remote sites. So let’s create a contact list to store client and key contact details. From “Site Actions”, choose “Create” and from the list of choices, create a new Contacts list.

We will call this list “Clients” and add some clients in.

“Client Sites” list

Next up, each client can have one or more sites. For each site, we are interested in not only its physical location, but also the IP Address range used at that site, the VPN type and detail. For this we will create a custom list called “Client Sites” with the following columns

Now we add some sites and assign them to the aforementioned clients. Note that in our example, ABC Incorporated has two sites. Since the Client column is a lookup against the Clients list we previously created, the “Client” column below is linked to its associated entry in the “Clients” list.

“IT Assets” List

Right! We have site information with some basic information about VPN details.How about we now create an IT Asset/Device list where we record the details about our equipment at these sites. In this list, we are interested in the type of device, its IP Address, serial number, purchase date and warranty.

Below is the configured list and some sample data

Examining the list relationships

So we now have three lists containing contact information, site information and device information. Any column that is of type “lookup” is automatically linked to the source of the lookup. So in the sample above of the device list, clicking on “Milton Keynes”, takes you to the entry in the “Client Sites” list as shown below.

From here, we can example the details of the client for this site, “ABC Incorporated”. Clicking on the client reveals our contact details for that client.

So what have we achieved so far? We have taken some useful information that would usually be stored in different locations (and potentially formats/systems) and linked it together. Being web based, the information is accessible from anywhere on the network. Remember, SharePoint also has a built in recycle bin and full version history if you wish to use it.

That is all well and good, but we have not linked this yet to our Cisco backups. In addition, we want to add our SOE documentation to be part of this also. So let’s create a new document library called “IT Asset Documentation”, and relate this to the devices.

IT Asset Documentation Library

So this time, we create a document library instead of a list. Here are our columns

So, now we create or upload our SOE, Network Diagrams and Maintenance Agreements so this document library as shown below.

In case you haven’t seen it before, Office 2007 is aware that you are using a SharePoint document library and allows you to set the values of these columns from within the application itself as shown below.

By creating the “Related Device” field, we now have a direct link to the three lists created earlier in this post. In addition, we have the ability to sort, filter and group documents in this library based on the columns we have added. So for example, here is another view of the document library grouped by document type.

For Cisco people reading this, notice how this view now looks a little like folders based on the document type? The importance and usefulness of views should not be underestimated. Unlike folders, views allow you to sort/manage your files much more flexibly than a static, folder structure. A single document library or list can have many views!

Finally, the configuration backups!

So if you were observant earlier, you would have noticed that I created a fourth document type called “Configuration Backup”, that we have not yet used. Let’s now integrate the TFTP server into this new library.

The simplest way that requires no customisation is to set the default document type to be “Configuration Backup”. That way, any file that is TFTP dumped into this library will be assigned that document type. So let’s modify the “Document Type” column and make this change.

It doesn’t really matter which TFTP method you use, so long as it is configured to drop files into this document library. If you use one of the TFTP servers used in part 2, then in this example the WEBDAV UNC version is:

• \\tidemo\tftp\IT Asset Documentation

And the URL for the web service/sdk method (as described in part 3 and part 4) is:

• http://tidemo/tftp/IT%20Asset%20Documentation/

I used the TFTPUtil method with web services and uploaded a file. Below is the document library with the new file. Note the document type!

Improving Integration with Workflow

Let’s now do a really simple workflow example to make this a little slicker. How about when a new configuration file has been TFTP’d into this library we perform these steps:

1. Check if the ‘related device’ field is empty
2. If it is empty, create a task for the network administrator to add the device details to the “IT Asset” list and link it to the uploaded file.

Since we have version control enabled in this library, the first time a file is uploaded the workflow will fire, and set this task for the network administrator. Once the related item is set, subsequent uploads will trigger the workflow, but it will do nothing because the related device has already been set.

I’ll create this workflow using SharePoint Designer. SharePoint designer workflows are by no means the answer to global warming but they are a great method for rapid workflow development with no programming expertise required.

In SharePoint designer we open our site and create a new workflow from the File Menu…

Name the workflow “Check Related-Device Field” or something similar and attach the workflow to the list “IT Asset Documentation”. Make sure the box to “Automatically start this workflow when a new item is created” is ticked.

Click NEXT and we add our first and only workflow step. The first thing to do is add a condition.

The first condition is always to compare the value of a field in the list the workflow is assigned to “IT Asset Documentation”. Click it, and choose the “Related Devices” field for the “field” parameter. Click the “equals” parameter and choose “is empty”.

Now we will create our action to assign the task to the network administrator. Click the “Actions” button and choose “Assign a To-do Item”.

Now in the newly created action, click “a to-do item” and on the next screen create a task name called “Update Cisco Config Documentation” and give it an appropriate description. Then Click “Finish”

Finally, we assign this to-do to the network administrator by clicking the “these users” link and finding the network administrator in the address book (Active Directory).

So here is the completed action.

Click “Finish” and we are all set!

Testing it out

So, let’s test by uploading a new file.

C:\>tftp 192.168.134.129 put c:\myfile.txt ABCRTR03
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

And the document library…

WOHOO! You can see a new column has been created to track our workflow status.

The status is “In Progress”, because it is waiting for a task to be completed. If we examine our SharePoint site, we have a new list, called … wait for it … “Tasks”!!

Let’s take a look at it!

If you click on this task (which also can be viewed and edited in Outlook), we can view this exciting task.

So our network administrator has received this task and has dutifully created a new entry for this router in the “IT Asset” list. They have then updated the “Related Device” column in the “IT Asset Documentation” as shown below

So now our intrepid network admin can edit this task and mark it as complete! Click da button!!

Now that the task has been completed, the workflow has completed and this is reflected in the “IT Asset Documentation” library as shown below. Note the workflow status is “completed”. Cool!

Conclusion

So what have we achieved here? This is a really basic example of improving configuration management, streamlining a process, improving the quality of documentation and enforcing accountability because we now have an audit trail of the task being assigned and completed.

We really are scratching the surface though. We have the tools here to implement much more that just making sure that something is documented.  In the next post, we will expand on this site and incorporate a change control mechanism using forms services. We will also start to look at this sort of stuff from a higher level, strategic viewpoint in the context of the ‘greater’ IT infrastructure management.

In conclusion, I hope that you Cisco guys found this post useful and look out for part 6 of this series in the near future.

Cheers

Paul

                      

No Tags

Welcome to part 4 of my series on demonstrating SharePoint’s usefulness for storing Cisco configuration backups. What a hard slog it’s been! The last article (part 3) of this series focused on how to modify an open source C# TFTP server to upload files into a SharePoint document library using the SharePoint SDK.

Here is the quick recap on what we have covered so far

• Part 1 illustrated how it it possible to use SNMP to tell a Cisco IOS device to dump its configuration to TFTP. We talked about the version control feature of SharePoint and why it makes sense to TFTP your configs to a SharePoint document library. We covered the WEBDAV network provider supplied with XP and Win2003 and finished off with a basic example using the TFTP server TFTPD32.
• Part 2 went into more detail about the issues you can face when using the WEBDAV network provider. It also went into more detail on 3 TFTP server products (WinAgents TFTP, SolarWinds TFTP Server and TFTPD32 and why TFTPD32 ended up being the best choice for this purpose.
• Part 3 then looked at a wonderful open source TFTP server written in C# called TFTPUtil. We modified the source code of this TFTP server to use the SharePoint SDK and upload files to a SharePoint document library.

Now both the WEBDAV and SDK methods had some issues. The WEBDAV method was obviously easy to set up because pretty much any application (theoretically anyway) can be made to work with it. I, however, had reliability issues with this method as part 2 detailed. The SDK method was much more reliable, but had its own problems. Many people would be uncomfortable with having to perform custom modification of an existing open source TFTP server, but more importantly, there are security implications with this method too.

So we have one remaining method that we can explore. This method is still based around modifications to the TFTPUtil source code but instead of using the SharePoint SDK, we instead use the SharePoint web services.

SharePoint 2007 Web Services

SharePoint can be remotely interacted with via Web Services. Out of the box, a number of web services are provided and shown below (along with the URL of the service based around my sample SharePoint server used in this series).

• Administration Web Service (http://tidemo/_vti_adm/Admin.asmx)
• Alerts Web Service (http://tidemo/_vti_bin/Alerts.asmx)
• Authentication Web Service (http://tidemo/_vti_bin/Authentication.asmx)
• Copy Web Service (http://tidemo/_vti_bin/Copy.asmx)
• Document Workspace Web Service (http://tidemo/_vti_bin/Dws.asmx)
• Forms Web Service (http://tidemo/_vti_bin/Forms.asmx)
• Imaging Web Service (http://tidemo/_vti_bin/Imaging.asmx)
• List Data Retrieval Web Service (http://tidemo/_vti_bin/DspSts.asmx)
• Lists Web Service (http://tidemo/_vti_bin/Lists.asmx)
• Meetings Web Service (http://tidemo/_vti_bin/Meetings.asmx)
• People Web Service (http://tidemo/_vti_bin/People.asmx)
• Permissions Web Service (http://tidemo/_vti_bin/Permissions.asmx)
• SharePoint Directory Management Web Service (
• Site Data Web Service (http://tidemo/_vti_bin/SiteData.asmx)
• Sites Web Service (http://tidemo/_vti_bin/Sites.asmx)
• Search Web Service (http://tidemo/_vti_bin/spsearch.asmx)
• Users and Groups Web Service (http://tidemo/_vti_bin/usergroup.asmx)
• Versions Web Service (http://tidemo/_vti_bin/Versions.asmx)
• Views Web Service (http://tidemo/_vti_bin/Views.asmx)
• Web Part Pages Web Service (http://tidemo/_vti_bin/WebPartPages.asmx)
• Webs Web Service (http://tidemo/_vti_bin/Webs.asmx)

So let’s go back to our TFTPutil project in Visual Studio and use Web Services instead of the SDK

Modification Example #2 Web Services

Initially I thought that I was going to have to write my own web service to upload a file to SharePoint, but like TFTPUtil, it didn’t take much googling to find that the hard work had already been done.

In this case, it was the excellent work by txs8311. He had written some articles on how to upload into SharePoint via methods provided by the out of the box web services. We will incorporate his work into TFTPUtil and see how it goes!

Out of respect for txs8311, I will not re-paste his code here, but I will show how I have incorporated it into TFTPUtil.

If you check his blog post, he has created a class called DocLibHelper. We will add this as a class to TFTPUtil and then call it from the TFTPServerProcess class that we modified in my last post.

So let’s go back to our TFTPUtil project in Visual Studio and add a new item to the project. Add a class and call it DocLibHelper.

\

Once added, paste the content of the DocLibHelper class from txs8311′s site. Do not forget to add the references to the libraries used by the code.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;
using System.Xml;

Finally, the code interacts with the Lists web service (lists.asmx) and we need to add a reference to a SharePoint web service so we can compile and test the code against the lists web service. From the “Project” menu, choose “Add Web Reference”. In the URL, connect to the lists service for your SharePoint server (in my case http://tidemo/_vti_bin/Lists.asmx)

You MUST name this Web Reference ListsService as txs8311′s code expects it to be this name.

That’s it for the webservice code! Now let’s modify the TFTPServerProcess class to use it! (I go into detail on modifying this class in the previous post).

State.Close();
if (IsSharePoint){
    // Now we add this newly downloaded file to the document library
    try
    {
        DocLibHelper docLibHelper = new DocLibHelper();
        Dictionary<string, object> properties = new Dictionary<string, object>();
        properties.Add("Title", State.Filename);
        string uploadURL = TempPath + @"/" + State.Filename;
        //Create or overwrite text file test.txt in 'Docs' document library creating folder 'Test Folder' as required.
        docLibHelper.Upload(uploadURL, System.Text.Encoding.ASCII.GetBytes(FullPath + @"/" + State.Filename), properties);

        // clean up temp file
        File.Delete(FullPath + @"/" + State.Filename);
    }
    catch (Exception ex)
    {
        AddMsg(Level.Verbose, "Problem opening SharePoint site" + TempPath);
        AddMsg(Level.Debug, "Exception: " + ex.Message + "\n" + ex.StackTrace);
    }
}
// This is the last block so go ahead and setup to delete stateStopListener(); 
StopListener(); 

So let’s have a closer look at this code. Now we are invoking an object based on txs8311′s the DocLibHelper class, rather than the SharePoint SDK classes. After creating an instance of DocLibHelper, we create a dictionary object to hold the properties of the file. We will write the name of the file to the “Title” column (the result of which you will see later).

We then call the Upload method, passing in the URL, a reference to the file to be uploaded and the properties dictionary.

One little gotcha!

One minor thing that I have to mention here, as I got caught by it. Web Services are case-sensitive! The invocation code on the console application was the following:

TFTPServer myTFTP = new TFTPServer();
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.Path = @"http://tidemo/tftp/backups";
myTFTP.StartListener(); 

But in SharePoint the document library is actually called “Backups”. See the problem? I kept getting exception errors telling me “List not found”, and it took me a while to twig to the fact that I had to fix up line 4 of the above code.

myTFTP.Path = @"http://tidemo/tftp/Backups";

Now that oughta do it! Let’s test!

Now we attempt an upload.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Sweet!

Let’s check the document library. First up view properties so we can see the title field. Very good.

Okay now let’s do a few more TFTP PUT’s.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip]

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

So let’s look at the version history. What do you see now? SharePoint will report changes to properties (column values) in the version history. Version 285 of myfile.txt was the version where I updated the title property.

This method works nicely. The only problem I encountered was a delay after the first running of the program. To connect to the lists webservice, authenticate and call the necessary methods a little while on the first attempt. But subsequent attempts were fine. Since I did this on a VM on a heavily loaded laptop, it may not be noticeable in a prod environment at all.

Modification Example #3, FrontPage Extensions and RPC

txs8311 also wrote a second part to his posts on uploading files into SharePoint. In this post he rewrote his DocLibHelper class to use FrontPage extensions and RPC. I did not try this, as I was satisfied with the modification Example #2. But all you should need to do, is replace the code in the DocLibHelper class with the code from his page.

If you are more keen than me, feel free to give it a shot and let me know how you go!

Summary of Upload Methods

So let’s recap on the three TFTP upload methods that we have used and summarise the advantages and disadvantages of each.

CleverWorkArounds Rating: Rock on SharePoint Web Services!

Additional Coolness

Since we have modified this TFTP Server to our requirements, you are pretty much free to add SharePoint specific features to make integration even nicer. One feature that springs to mind is setting more metadata properties when the files are uploaded. Here is an example. Let’s assume that we have a list of Cisco IP addresses and the MODEL and TYPE of device on that address. E.g 6509 Switch, 3725 Router or Aironet 350 Wireless Access Point.

It would be pretty easy to modify our TFTP server to be able to examine this list when receiving a file and based on the IP of the sender of the file, write the type of device to a “Device Type” column in our configuration backups library.

Better still, it would then be very cool if that list of IP’s and device types already exist in SharePoint. Imagine this list exists in the form of an IP address register or better still, an IT device asset register (along with serial numbers, maintenance contract information and the like). It would be easy for staff to maintain this information and the list would have version control just like our “Backups” document library. But for me what would be really nice about this method is the organisational efficiency that would be gained.

That gain is this: The person that updates the asset register list isn’t necessarily the Cisco engineer that sets up the device. Critical device information is entered ONCE and leveraged from then on. The next post in this series will expand on these benefits.

Conclusion

Right! We are DONE with getting files into SharePoint and about damn time too!

The next posts in this series will move away from application development and infrastructure issues and move into some higher level governance issues. The aim here is to show Cisco and IT infrastructure people what you can do with SharePoint once you get the data into SharePoint. I hope if I do a decent job of writing it, that it will be easy to see just how this can be re-used for any IT infrastructure management.

But for now, I bid you adieu until then….

                      

No Tags

As I write this series, it is getting less and less about Cisco and more and more about SharePoint. This article is definitely developer centric, but since Cisco guys tend to be interested in the guts of the detail, I decided to keep going .

If you read my articles I tend to take the piss out of IT role stereotypes just to make it more entertaining reading. Sales guys and IT Managers tend to cop it the most, but I also like to have a dig at the expense of the nerds too. Cisco nerds on the whole are a great bunch, but I have to say, the scariest nerd I have ever met drank Cisco kool-aid in jumbo size!

If you have gotten to this article after reading the first two and you are scoffing at my audacity to suggest you TFTP your configs into SharePoint, chances are most people think you’re scary! If you are hitting this series of articles for the first time, go back and read part 1 and part 2 before being scary!

Seriously now, I thought that this would be a 2 part set of articles, but I got all bogged down in the methods of getting files into SharePoint. The WEBDAV based methods described in the previous article is easy to do, but ultimately is not the recommended method. So now, we will look at the ‘proper’ ways to do it and see if they are worth the effort. They work okay, but are more complex and I’m not convinced that the governance issues are necessarily worth it for many readers.

Degree of difficulty for this article is varied.

CleverWorkArounds Coffee requirement rating (for an application developer):  

CleverWorkArounds Coffee requirement rating (for a non developer):    

How can we we *not* use WEBDAV?

If we are using methods other than WEBDAV, then how are our existing applications supposed to work? I mean, all they know is the windows networking API. So, unless the other methods are implemented as a redirector (as described in more detail in the “WEBDAV Interlude” section of my first article), how can they use any other API?

The answer is, they can’t! So we are leaving the world of freeware and cheap TFTP servers and writing our own one!

“What? I’m no coder”, you say? That’s okay, nor am I… but stick with me anyway.

Okay, how about we modify someone else’s open source TFTP Server to work with SharePoint via non WEBDAV methods?

Fortunately for us, such a TFTP server does exist. In addition to being conveniently modifiable (got to love open source), it is multithreaded, very well written and easy to understand and extend.  Kudos to the author. If you write a few scripts here and there, it should be enough for this article.

So let’s take a closer look. If you already know visual studio you can skip the next section and move to the section entitled “Examining the Code”. Otherwise let’s keep going getting this TFTP Server compiled!

Getting TFTPUTIL compiled

Head on over to http://sourceforge.net/projects/tftputil and download the latest release. Crank out Visual Studio 2005 and take a look at it.

This server is implemented as a class, so you have to write your own little program to make use of this class by creating a TFTP server object. You do by adding a new project to your Visual Studio workspace. But before we do that, how about we deal with a pre-requisite component.

The TFTPUtil class makes use of a library of useful functions called NSpring. You will want to download this. Grab the nspring.dll, and then tell Visual Studio how to find it by choose “Add Reference” from the “Project” menu, . At the resulting dialog box, go to the “Browse” tab and navigate to where you downloaded nspring.dll. (Best to put into C:\WINDOWS\SYSTEM32 or use the same folder where the TFTPutil code lives).

Right, we are all set! You could compile and build TFTPUtil.DLL right now, but it would not do you a lot of good because it’s a class and does not run on its own. It needs to be *invoked*. This means we need to create a little application that creates an *instance* of the TFTPUtil class. Right click on the project and choose “Add New project” as shown below.

For our purposes, we create a new “Console Application” project and call it myTFTPServer or something meaningful to you. This will be a small EXE file that uses the TFTPUtil class to create a TFTP server.

Now, right click on the newly created project and choose “Set up Startup Project”

Next step, we need to tell this new console application project how to find the TFTPUtil class. From the Project menu, choose “Add Reference”. At the resulting dialog box, go to the “Project” tab and you will see the open TFTPUtil project.

Right, that should be it! Your Visual studio should look something similar to this

Creating a basic TFTP server

This is a well written TFTP server. Everything is very nicely modular and easy to understand without getting into too much complexity.

So first up, here is the 5 lines of code of your console application to get the TFTP server working. Add this to the code section of the static void Main(string[] args) method.

using TFTPUtil;

TFTPServer myTFTP = new TFTPServer();
myTFTP.Path= "C:\TEMP";
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.StartListener(); 

Those 5 lines create a new TFTP server object, set the destination folder, allow it to receive files, overwrite received files and start the TFTP server. If you now compile and build this solution you will end up with the files TFTPutil.DLL and myTFTPServer.EXE.

You should be able to test it now. (By default, files will be uploaded to the folder the EXE is run from unless you specify a path).

Examining the code

Now let’s look at the area of the TFTP server where we need to add some code. It is in TFTPServerProcess.css in a method called “ProcessDataGram”

If you examine this method, you will see the following code (just search for it):

AddMsg(Level.Info, "Successfully received file " + State.Filename + " (" + State.Filesize.ToString() + " bytes) from " + State.RemoteIPAddress + " in " + TotalTime.ToString() + " seconds (" + (Rate).ToString() + " bytes/sec).");
//This is the last block so go ahead and setup to delete state
State.Close(); 

We will make some fairly basic modifications to the server in this section, so that the file is uploaded to SharePoint.

There are several methods that can be used to get files into SharePoint. We will do this first by using the most obvious way: The SharePoint SDK.

Modification example #1 – The SharePoint SDK

The SharePoint SDK is definitely the easiest way to upload a file, but there are several caveats:

• The TFTP server MUST run on a server that is a member of the SharePoint farm
• The TFTP service account MUST have administrative privileges on the server and SQL access

That second requirement in particular just plain sucks. After going to the trouble in part 1 to set up low privileged account “tftp service” with “contribute” access to a document library, we now find that we need to grant the account significantly more privileges than before. My security nazi Cisco readers will read that and say “no way will I do that”, and I would concur. But fortunately for my readers (and unfortunately for me) I only discovered that caveat AFTER I had gotten it working. So I will show you the code changes anyway.

So, first up I will tell you what we will do. The aforementioned ProcessDataGram uses a class called State that is used to write the file to the file system. Then later the method Close() closes the file that has been uploaded.

So what we will do, is upload the file to SharePoint just after State.Close(). This means the file is still downloaded to the local file system and then copied to SharePoint. Therefore, the local file system is now simply a temporary place for the file to be stored, prior to it be uploaded to SharePoint.

To do this, we will change how TFTPUTIL handles the PATH property. Recall in my basic TFTP console application example code I could set the path via:

myTFTP.Path = "C:\TEMP";

So let’s change this behaviour so that if Path contains the string “http://”, we assume it is a SharePoint path, rather than a file system path. Therefore we set the internal file path to %TEMP% so we can still capture the file, but then we will upload it to the http:// specified SharePoint site.

Path is a property of the TFTPUtil class that is defined in TFTPServer.cs. We will make a minor change to the Path property, because it performs a check that will always fail when a SharePoint URL is supplied.

public string Path
{
    get
    {
        return FullPath;
    }
    set
    {
        if (value.Contains("http://")) {
            FullPath = value;
        } else
        {
        if (System.IO.Directory.Exists(value))
            FullPath = value;
        }
    }
} 

Now we turn our attention to the TFTPServerProcess.cs file where we have to modify the TFTPServerProcess class. We will add two more properties to the TFTPServerProcess class. Find the declaration for FullPath and then add the two lines in bold below it.

private string FullPath = System.IO.Directory.GetCurrentDirectory();
private string TempPath = System.IO.Directory.GetCurrentDirectory();
private bool IsSharePoint = false; // If this is a sharepoint site 

Also while we are here, add a reference to the SharePoint dll.

using Microsoft.SharePoint; 

You will find the Microsoft.SharePoint.DLL in C:\Program Files\Common Files\Microsoft Shared\web server extensions on most installations.

Next, we need to add some basic logic to the code where the TFTPServerProcess class is initialised. It is an overloaded class so it is declared twice. To keep it simple, I have added the code to each declaration. If this is a Sharepoint path, it copies the supplied path to a variable called TempPath, and then puts the path of %TEMP% into the FullPath. This is deliberately done so that minimal code changes elsewhere are required (ie the rest of the class expects a variable called FullPath to point to a filesystem, not SharePoint).

Thus, it looks like this (bold):

Declaration 1:

public TFTPServerProcess() 

if (FullPath.Contains("http://"))
{
    IsSharePoint = true;
    TempPath = FullPath;
    FullPath = Path.GetTempPath();
}

Declaration 2:

public TFTPServerProcess(string FilePath,
    Level LoggingLevel,
    Level DisplayLevel,
    bool AllowRRQ,
    bool AllowWRQ,
    bool AllowWRQOverwrite,
    bool AllowTFTPOptions,
    bool CheckReadWriteState,
    int ResendInterval,
    int TimeoutInterval,
    Logger logger,
    IPAddress srvaddr)
{ 

    if (FilePath.Contains("http://"))
    {
        IsSharePoint = true;
        TempPath = FilePath;
        FullPath = Path.GetTempPath();
    }
    else
    {
        FullPath = FilePath;
    }

Finally, we add some code just after the State.Close() method as shown earlier. This takes the downloaded file from %TEMP%, uploads it to SharePoint and then deletes the file from %TEMP%.

State.Close();
if (IsSharePoint)
{
    // Now we add this newly downloaded file to the document library
    try
    {
        SPSite DestSite = new SPSite(TempPath);
        SPWeb DestWeb = DestSite.OpenWeb();
        DestWeb.AllowUnsafeUpdates = true;
        SPFileCollection files = DestWeb.Files;
        FileStream destFile = File.OpenRead(FullPath + @"/" + State.Filename);
        string destFileURL = TempPath + "/" + State.Filename;
        Hashtable MetaDataTable = new Hashtable();
        SPFile destSPFile = files.Add(destFileURL, destFile, MetaDataTable, true);
        destFile.Close();
        // clean up temp file
        File.Delete(FullPath + @"/" + State.Filename);
    }
    catch (Exception ex)
    {
        AddMsg(Level.Verbose, "Problem opening SharePoint site" + TempPath);
        AddMsg(Level.Debug, "Exception: " + ex.Message + "\n" + ex.StackTrace);
    }
}
//This is the last block so go ahead and setup to delete state
StopListener(); 

Note the HashTable MetaDataTable line. Even though we are not setting any metadata values, I have left in HashTable code because it illustrates how this method allows you to code in the setting of metadata columns in a document library. For example, imagine that the SharePoint document library has a column called “DeviceType”. Your code could say, write the string “Cisco” or “Linksys” to the DeviceType column when the file is uploaded.

So let’s try it out shall we?

Modify your console application to fix the path variable to a SharePoint doc library and then build the solution. 

TFTPServer myTFTP = new TFTPServer();
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.Path = "http://tidemo/tftp/backups";
myTFTP.StartListener(); 

Test the solution and you get a fairly unexciting console window.

Now we attempt an upload.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip it all working swimmingly]

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Also, this time around, none of the locking/access errors seen with the WEBDAV based methods – WOHOO!. Running this as a service also works fine, using the same freeware utility I used in the last post (ServiceInstaller ). Just compile the solution and then use ServiceInstaller as I described for TFTPD32 in the previous post.

Remember what I said before though, the “tftp service” account MUST have the following privileges on the server:

• The user is a server farm administrator.
• The user has Read and Write permissions on the content database.
• The user is a site collection administrator.
• The user has permissions to access the Windows SharePoint Services site or the SharePoint Server 2007 site through which the code iterates.

Read more at KB Article 935751, but the fundamental issue is that we are not running this code as the IIS process, but rather a separate user account. “If code is executed from a custom Windows-based application or from a custom console application, the code path does not use the Web context that is hosted by the Microsoft Internet Information Services (IIS) application pool. Therefore, specific user permissions are required in SharePoint Server 2007 and in the back-end data source”

Conclusion

The requirement for administrator level privileges sucks. It is not a showstopper, but it sucks nonetheless. So the next post, we will turn our attention to the Webservices methods of uploading data and see if they are any better. Then (finally!) we will get onto more interesting topics like making use of workflow and integrating it with other useful information.

If you are wondering whether you can download my changes and test for yourself, not yet . I’m going to contact the author of the TFTPUtil project first, and I have to perform more debugging . So right now you will have to do it yourself.

Thanks for reading.

Paul

                      

No Tags

Here I am back again, illustrating some of the interesting possibilities that SharePoint offers for Cisco people.

To recap my last post, I showed you a little perl script I wrote to get an IOS router or switch to dump its current configuration to a TFTP server. I then used one of several freeware TFTP servers to show how you can have a TFTP server save the captured file into a version enabled document library.

I then hit a snag in relation to using a Windows Service to do this task. In this article we will delve into this issue in more detail. In addition, I ended up delving much deeper than I intended. So, like my branding series, this is going to turn into a multi-part series too, covering some application development, configuration, security and governance issues. How many parts it will end up being is anybody’s guess!

This is a technically oriented series of articles for the most part, so for you people who like the governance and finance stuff, you may not get too much out of this one. Although this article (part 2) focuses on my issues and observations with the Windows WEBDAV client, if you are one of these people who have ‘special’ feelings when you see those pretty blue Cisco boxes like the image above, then you may find some useful content here.

SharePoint developers and architects may also find this of interest.

Now I previously mentioned that I tried three TFTP servers in writing this article and had some problems. They were:

• WinAgents TFTP (Commercial)
• SolarWinds TFTP Server (Freeware – Registration required)
• TFTPD32 TFTP Server (Freeware)

And let’s not forget good old PUMPKIN, just to make one of my CCIE friends, who read the first post and suggested I use it as an alternative, happy (although he forgets that years ago I was the one who told him about Pumpkin in the first place )

Let’s review their capabilities and cover the issues encountered with each. As we go, we will get to see the uglier side of WEBDAV.

WinAgents TFTP

I’ve used this product several times in the past. At the time it was pretty much the only product that would natively run as a Windows service. It is not free, but buying it won’t exactly break the bank. It has a nice GUI, good logging and the ability to set up virtual folders to different underlying locations. Below is the main configuration screen and the screen where multiple locations can be configured.

You would think that the above ability to virtualise folders would make this product an obvious choice for front-ending to SharePoint, because it could co-exist with native filesystem, DFS and 3rd party SMB based resources like those provided by samba.

You would be right, but the latest version is let down by one stupid design flaw. When you send a file to a WinAgents TFTP server that already exists, it will delete the original file and then copy the new file to the virtual folder. It may on the surface seem no different to simply overwriting a file – but there is a big difference in a SharePoint site.

All files in a SharePoint document library have a unique ID. It is a built-in column and can be made visible by modifying a document library view as shown below.

By deleting the file in SharePoint, that file is now put into the recycle bin. When you upload a file of the same name, SharePoint treats it as a new file and allocates it a new ID. Therefore, there is no version history at all. Each upload is deleting the previous version and uploading a new file.

This obviously blows away any potential to use WinAgents TFTP as a SharePoint integrated TFTP server .

CleverWorkarounds Rating: They probably thought it was a good idea at the time.

Solarwinds

So instead, let’s now turn our attention to the SolarWinds TFTP product. This one is free and also very easy to use. Like the WinAgents product, it can be run as a service out of the box, but it is not as comprehensive as its commercial competitor in terms of logging, granular security filtering and folder virtualisation.

Unlike its commercial cousin, however, it does not delete files before uploading, and therefore plays nice with SharePoint version control. That, combined with natively running as a windows service, makes it the obvious contender to be the SharePoint TFTP server of choice.

The GUI is basic yet functional. There is a status screen and a configuration screen that allows you to set the TFTP Root directory as shown below.

In the screen-shot above, you will notice that the TFTP Server root directory is \\tidemo\tftp\backups. As I explained in the first post, this is the WEBDAV UNC equivalent of the SharePoint site http://tidemo/tftp and the document library called backups.

If you recall in the first post, we created a user account called ‘tftp service’. This account has been granted access to the http://tidemo/tftp/backups library. So, we need to set the credentials of the WinAgents TFTP service to run as that user. Therefore, when files are received by TFTP clients, the server will be able to write to the library. Below is my screen capture of my configured TFTP server.

There was one other task I also had to do to test this setup. SolarWinds is a little bit naughty in that it logs to its installation folder. (‘tsk ‘tsk will never get a designed for XP gold logo). Being good practitioners of IT security, the ‘tftp service’ account is a domain user with no system privileges, and only has READ access to this folder by default. Changing configuration parameters (such as the UNC TFTP root) also would not be saved, because the TFTP server would not be able to write to its own configuration file when running as the ‘tftp service’ account..

So the permissions of the SolarWinds installation folder has been modified so that the ‘tftp service’ account has WRITE access as shown below.

So I started the service fine, and ran a test! Wohoo we have our file!!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

So I tried it again a few times and hit a snag after some successful retries…

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\TFTP 192.168.134.129 put myfile.txt
Error on server : The process cannot access the file ‘\\tidemo\tftp\backups\myfile.txt’ because it is being used by another process.
Connect request failed

This was odd. Suddenly I was unable to copy a new version of the file to TFTP. I was running many repeat tests, so perhaps there was some sort of temporary file lock causing a problem? So I waited for a bit and then was able to transfer successfully again as shown below.

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

But it wasn’t long before I got another and more severe error.

C:\TFTP 192.168.134.129 put c:\myfile.txt
Error on server : The network path was not found.

Connect request failed

The TFTP server log showed this error and ominous stack trace…

2008-01-23 21:49:53,102 [6] WARN  TFTPServer.Service.TransmissionPut – Exception caught while trying to open file for reading.
System.IO.IOException: The network path was not found.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access)
   at TFTPServer.Service.TransmissionPut.Begin(RequestPacket firstPacket)

Network path not found? What the hell was going on here? It worked a minute ago!

Eventlogs came up blank, and I tried an IISRESET. Still no-go. Restarted the TFTP service and I was able to transfer again. However, it didn’t take long for the problem to re-appear.

I tried all of the suggestions on Mark Muller’s blog article with no success.

I found two sites of interest and one seemed close to this issue. The first was here, where a developer was having the identical problem, and a Microsoft employee suggested not to rely on WEBDAV “especially when using services”. The second one was a confidence building quote from Microsoft themselves. They made an admission in their WEBDAV Whitepaper regarding the Web Client service. “The Web Client Service in Windows XP and Windows Server 2003 does not function correctly if you stop it then start it again without restarting the computer. Although a bug has been opened on this problem, the changes required to fix it are too large for a hotfix or a service pack and have therefore been delayed until a future major release of Windows”

Are you freakin SERIOUS!?!

Anyway, I decided that before I go off on a Microsoft bashing rant, I’d better double check that this problem was with SolarWinds TFTP or the obviously rock solid and stable WEBDAV client.

CleverWorkAround Rating (SolarWinds TFTP) – Judgement reserved till later

TFTPD32 TFTP Server

TFTPD32 is more than just a TFTP server. It also provides DHCP, Syslog and SNTP services as part of the deal. Unlike the previous two TFTP servers, it does not run as a windows service out of the box, but it can be easily made to.

Cisco have also decided that this is their recommended TFTP server as well, not a bad endorsement, eh!

If we focus on the TFTP server functionality only, it is basically the same as SolarWinds without the windows service bit. So WinAgents still wins for overall ease of use and granularity but as discussed it was a waste of time with SharePoint.

Once again there is a main status screen, showing you current activity, as well as a configuration screen where you set the TFTP server root UNC path.

So given that we had all of the WEBDAV stability problems with SolarWinds, this TFTP server allows us to test two scenarios. Since it can be run as standalone, we can run it interactively as our ‘tftp service’ account and test it. Then, we can use a 3rd party tool to make it run as a Windows Service, and compare notes.

So first up, let’s run it interactively as rtftp service’ account. This is achieved by RIGHT clicking on the TFTPD32.EXE file and choosing RUN-AS from the menu options. When prompted, enter the credentials of the ‘tftp service’ account.

If you want absolute assurance that TFTPD32.EXE is being run by the ‘tftp user’ then look for the process in “task manager” and check the “User Name” column.

So, here we go!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Now occasionally I received the error below, but unlike when using SolarWinds, running as a windows service using ‘ttp service’ user credentials, it would always happen once and subsequent attempts would be fine.

C:\TFTP 192.168.134.129 put myfile.txt
Error on server : Access violation
Connect request failed

Just to prove that it was much more reliable, we now look at the version history in SharePoint for the file myfile.txt

CleverWorkArounds Rating: Not flawless, but reliable

So we have ascertained that running TFTPD32 works pretty well when run interactively as ‘tftp service’ . Now we have to run TFTPD32 as a service and compare

TFTPD32 as a windows service

There are many utilities out there to run a regular EXE file as a Windows Service. One of the most popular ones is SVRANY from the Windows 200x resource kit. However I used the freeware ServiceInstaller because it had a nice GUI

Using it is mind numbingly simple. Run the application and fill in the form as shown below and click the ‘Install’ button. Once installed, you can then change the service credentials to ‘tftp service’ the same way we did it for SolarWinds.

Start the service, and check in Task manager that TFTPD32.EXE is running. (When running as a service you will not see the status screen).

The moment of Truth!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Like when running interactively, I still received the occasional  ‘Access Violation’ error but always recovered. Again to prove that I ran lots of TFTP commands, the version history is now up to 204 .

CleverWorkArounds Ratings (TFTPD32 as a service) – The best of the WEBDAV TFTP bunch

CleverWorkArounds Ratings (SolarWInds) – Looks like it might be part your fault after all!

CleverWorkArounds Ratings (WEBDAV) – It’s definitely your fault!

Conclusion

So after these tests, it seems that running TFTPD32 as a Windows Service is no less reliable than running it interactively. Perhaps then there is an issue with the way that SolarWinds use the windows networking API’s to write files. SolarWinds is a .NET application and TFTPD32 isn’t. Therefore, they may well be using different API’s under the hood.

I wasn’t satisfied with all of this though. Although we have achieved our goal of TFTP to a version controlled document library, the whole unreliable WEBDAV thing left me with a bad taste and I decided to go down a different path. What more could I do, you ask? How about write my own TFTP server?

The next 2 posts in this series will cover getting TFTP files into SharePoint via *non* WEBDAV methods. We will examine their reliability and examine a little of SharePoint web services and the SDK.

The fifth article will leave the issues of getting the files *into* SharePoint, and instead focus on what we can do with them once they are there! (Workflows, metadata and the like).

The final article (I suspect it will be the final anyway but i won’t know ’til I write the next two), will tie it all together in terms of governance. Thus, it will be a little similar to the branding series where the last article was largely non technical in focus and audience.

I hope you’ve enjoyed this little journey and didn’t find it too pointless

                      

No Tags

Cisco nerds! This series is just for you! I know that you think you’re way too cool for collaborative portals, especially a Microsoft one at that. Instead you are more interested in delving into the IOS command line, to perform arcane arts such as debugging that OSPF route redistribution into BGP or getting off on planning and implementing a large scale DMVPN solution. Maybe you’re into QOS and VOIP and simply dig all of those DSCP-COS mappings, class and policy maps and the like.

Although packets, cells and frames are your world, *nix is cool, Nagios is your idea of a portal and anything remotely connected to Microsoft fills you with contempt and is beneath you right?

Well if this is you, I do understand your point of view because I was you once, but after some therapy, I’m now out of rehab and doing just fine!

Having Cisco/general networking expertise will help you with this article, so depending on who you are, the amount of caffeine required to follow this will vary:

CleverWorkArounds Coffee requirement rating (for a CCNP or CCIE):  

CleverWorkArounds Coffee requirement rating (for a non Cisco person or CCNA ):  

Okay Cisco gurus, if there is anything about SharePoint that you guys need to know about, it is the power of the document library.

Unlike a regular file system, a SharePoint document library has a powerful version control mechanism that can make your life easier, as well as play nice with the rest of your colleagues (whom you despise because they don’t understand or appreciate you ).

So I am first going to show you an example of how you can leverage SharePoint to do something quite useful with Cisco gear, and then in the next post of this series I will expand further into general IT infrastructure admin benefits.

The Cisco bit..

(Non Cisco people, your eyes will glaze over – but try anyway!)

Back around 2000/2001 when I was a full time network manager, I wrote a perl script that used SNMP to instruct a router to dump its configuration to a TFTP server. I had a cron job that ran this script daily, and it would tell all of our switches and routers to write their current configuration to TFTP. It was nothing special – many Cisco network professionals do the same thing.

(Before you ask, I *know* there are tools and new methods to do this nowadays, but back then the choices were more limited)

Anyway, here is a snippet of the script in question. I know it’s not great but it did the job…

use Net::SNMP;

($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localtime;
@Days = (“Sunday”, “Monday”, “Tuesday”, “Wednesday”, “Thursday”, “Friday”, “Saturday”);
$day = $Days[$wday];

# Now call the procedure to do the deed!
&backup_configs(“192.168.16.1″,”RWSNMPcommunity”);
&backup_configs(“192.168.16.2″,”RWSNMPcommunity”);

The above code simply grabbed the date and then called the main function, passing the IP of the router and the SNMP community string for Read/Write access. The routine below, used SNMP SET commands to tell the router the location of the TFTP server, the name of the configuration file being written, and then told the device to execute the request.

sub backup_configs {
    ($ipaddr, $cs) = @_;
    ($s, $error) = Net::SNMP->session( -hostname  => $ipaddr,
                                -community => $cs,
                -debug => ’1′ );

    $hnh = $s->get_request(“.1.3.6.1.2.1.1.5.0″);
    $hostname = $hnh->{“.1.3.6.1.2.1.1.5.0″};

    $filename = “\\router-configs\\” . $day . “\\” . $hostname . “.cfg”;
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.2.667″, INTEGER, 1);
    # Set the source to startup-config
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.3.667″, INTEGER, 4);
    # Set the destination to tftp
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.4.667″, INTEGER, 1);
    # Set TFTP IP Address
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.5.667″, IPADDRESS,”192.168.18.5″);
    # Set destination file name
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1x.1.6.667″, OCTET_STRING,”$filename”);
    # Set it off!
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.14.667″, INTEGER, 1);
    $done = false;
    $counter = 0;
    while ($done eq false) {
        $check = $s->get_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.10.667″);
        if ($check->{“.1.3.6.1.4.1.9.9.96.1.1.1.1.10.667″} == 3) {
            $done = true;
        }
        $counter++;
        if ($counter > 500) {
            $done = true;
        }
    }
    if ($counter > 500) {
        print “$date:A problem occured – it looks like the attempt failed\n”;
    }
    # Clear the config!\n”;
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.14.667″, INTEGER, 6);
    #close the snmp session
    $s->close;

}

Now, I know there is a lot of ugly SNMP stuff in my above code and many readers are thinking, WTF!? You can safely ignore most of the code. However there is one bit that I do want to bring your attention to.

    $hostname = $hnh->{“.1.3.6.1.2.1.1.5.0″};

    $filename = “\\router-configs\\” . $day . “\\” . $hostname . “.cfg”;

Looking closely at this, you can see that I am setting the filename based around the name of the Cisco device (retrieved from SNMP) and the day of the week the script is run ($day). This is because when I backed up the configuration of the router, I wanted to keep multiple versions. For simplicity, I opted to keep 7 days worth of configuration dumps. So by using the day of the week in the file path, I ended up with config backups like this.

• router-configs/monday/mycorerouter.cfg
• router-configs/monday/myedgerouter.cfg
• router-configs/tuesday/mycorerouter.cfg
• router-configs/tuesday/myedgerouter.cfg

So, provided the path router-configs/<day of the week> exists on my TFTP box, I end up with a single file for each cisco device in each day of the week directory.

Neat, eh?

Yeah it’s neat, so why integrate TFTP with SharePoint?

Version control and workflow, my skeptical Cisco friends, that’s why! The easiest way to explain is to dive right into an example. So let’s do that right now!

There are a few TFTP servers out there for Windows (yeah we are using Windows – get over it). I used to use WinAgents TFTP server. Unfortunately though in writing this article, the most recent version of WinAgents TFTP does not work with SharePoint the way I want it to (if you run an older version it is fine). I also tried using SolarWinds freeware (registration required) TFTP server as it runs as a windows service. Unfortunately, I had some stability problems with WEBDAV when running as a service (not SolarWinds fault) which I will explain in the next post, so I ended up using TFTPD32 with a few modifications. (TFTPD32 is recommended by Cisco anyway).

What we will do is the relatively simple process of telling the TFTP server to drop files into a SharePoint document library via WEBDAV.

For reference, the TFTP server does not have to be on the same server as SharePoint when using the WEBDAV method.

The TFTPD32 Server Installation is straightforward. Once installed however, there are three additional steps that you need to perform:

• Create or nominate a user account to run the TFTP Service.
• Modify TFTP32 to run as a WIndows Service – using the credentials of a user account that has permissions to the document library that the TFTP server will dump the files to.
• Set the TFTP root directory to the UNC path of the SharePoint document library

I am assuming that you already have a SharePoint or WSS server. On the SharePoint side, we need to create a site and a document library to hold our configuration backups, grant the TFTP server permission to it and then enable version control on that document library.

So in my example, the user account that is going to upload files to SharePoint is called “tftp service“. I will refer to this account frequently in the rest of this post.

Also in this example, the SharePoint site is http://tidemo/tftp and the document library is called “backups“

http://tidemo/tftp/backups

Configuring SharePoint

(SharePoint people, I am covering the basics here – skip if you know it already).

In SharePoint, create a new, blank site to host the TFTP server files. Being a bastion of originality, in the example below I have named the site TFTP.

Now that the site has been created, we create a document library to store the files for the TFTP server. I have called it “Backups” and chosen not to use a default document template.

The next step, having created this document library, is to turn on version control. This is performed by choosing “document library settings” from the “Actions” menu.

In the version settings screen, choose to create major versions.

Finally, we need to grant our user “tftp service” write access to this new document library. The simplest way to do this is add “tftp service” to the members’ group of the SharePoint site. This will grant them ‘contribute’ access to the “Backups” document library. (This permission setting is just for demonstration purposes. Adjust permissions according to your specific requirements)

Right! Our doc library is ready! If at this point you open the document library via Windows Explorer as shown below, you will see that the UNC equivalent path for this example document library is \\TIDEMO\TFTP\BACKUPS

If you do not see your standard windows explorer window here and a UNC path, stop here! We need to troubleshoot the windows WEBDAV service. It might pay for you to skip to the “WEBDAV interlude section coming up)

One thing that people often overlook with SharePoint is that document libraries have this “windows explorer” view. What is so significant about this, you ask? It is the fact that you can reach any SharePoint document library via a UNC path. You can even map a drive to a SharePoint library too. Thus the following document library:

http://tidemo/tftp/backups

is accessible via

\\tidemo\tftp\backups

As a result of this functionality, pretty much any application that deals with UNC and mapped drives (such as ROBOCOPY) can happily copy files from your PC to a SharePoint library (with a large caveat I will explain later).

For educational purposes, let’s look at how windows explorer view is done. (If you are not interested, skip to the next heading where we configure SolarWinds TFTP.)

How is this done? A WEBDAV interlude

Now SharePoint does not actually provide support for SMB which is what ‘regular’ shared file systems use. The protocol that SharePoint supports is WEBDAV which stands for “Web-based Distributed Authoring and Versioning”. So, how is it that we can use Windows explorer, mapped drives and UNC paths to get to SharePoint files?

In Windows XP and above, Microsoft supplied a new network provider called the Windows WebDav client. (Yes, XP and above – Windows 2000 is not going to work this way for you). Also, Samba people who are reading this and thinking “ooh I can use smbfs to mount a SharePoint library to *nix”, sorry it ain’t gonna work because it is not SMB.

By creating the Web Client Service as one of the built-in network providers, it is automatically queried whenever you make connection attempt to a network resource (i.e. opening Windows explorer and navigating to a mapped drive or UNC path). You can see the provider by following these steps:

• On the Start menu, right-click My Network Places and then click Properties.
• On the top menu bar, click Advanced and then click Advanced Settings.
• Click the Provider Order tab.

You will see a dialog box with your provider order listed. Note that the Web Client Network (WebDAV) is last in the list by default.

Microsoft have a detailed whitepaper on how Windows Explorer view is implemented, and it is well worth a read. (I’ve previously blogged on quirks with it before.)

TFTP to SharePoint

So, now we know that on XP and above, any application that uses the windows networking API (interpret that as refer to a mapped drive or UNC path), can get data into and out of SharePoint. So let’s run the TFTPD32 and set it to use our document library.

Configuring TFTPD32

First up, grab TFTP32 and install it onto your chosen server. After successful installation run TFTP32.EXE from the installation directory.

Click OPTIONS and now set the path to the Windows explorer view path (UNC) of your site and document library

That is pretty much it! Now let’s test it!

Unfortunately, I don’t have any spare Cisco gear handy as Uncle Ebay helped me dispose of them once I stopped doing networking. So instead, I will use the built in TFTP.EXE utility built into Windows. This proves that a file can be saved to SharePoint via TFTP.

It works like this.

TFTP <hostname or IP> GET|PUT <filename>

eg

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

The output of the above command shows “transfer successful”. So let’s go and have a peek in our document library!

Sweeeet, we have a file called “myfile”.

So let’s go back to TFTP and re-run the command a few times.

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many re-executes]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Now let’s take another gander in the document library. Still the one file there. However, let’s now examine the version history for the file and see if there are multiple versions?

Sweeet, we now see that there are 7 versions stored for this file!

Conclusion (and the catch…)

So imagine that every day, you use the perl script example above to instruct your networking gear to copy their configuration to TFTP. Now you can simply use a standard file name convention (such as <hostname>-config.txt) do not need to worry about folder structure or dates within filename to ensure that multiple versions are maintained. You can go into the version history for a particular router configuration file and retrieve the one that you need.

Of course, all of your other important documents such as your network diagrams, SOE documentation, maintenance and support agreement details can also be placed into SharePoint document libraries and many other benefits are derived in addition to version history, such as search, workflow and easy offline storage.

But that is for the next post

Now for the catch … and it’s a biggie!

We of course, want to run our TFTP server as a windows service, so that the TFTP server does not have to be manually invoked.

Try as I might, I was unable to get a reliable TFTP service writing to SharePoint document library, when the TFTP server is running as a Windows Service! This majorly sucked and I was an unhappy camper!!

I have concluded that the WEBDAV client in WIn2003 is flakey. In the next post of this series, I will outline the problems that I had and how I eventually was able to come up with a semi-clever workaround to make it all work.

Bye for now!

Paul

                      

No Tags

First up, let’s finish off a little theory to go with the first post.

Other WIFI Security Measures

WEP and the much improved WPA should not be considered alone. Security in general requires a holistic approach to determine what functionality the business requires, what techniques can be used to provide that functionality and then assurance of the security of those features. Below are two additional techniques regularly used in conjunction with WEP or WPA to improve the security of a WLAN.

MAC Address Filtering

A common feature available with wireless access devices is the means to restrict access to only wireless devices that have their network (MAC) address configured in a filter list. Therefore, any device that tries to authenticate with a MAC address not matching the list will be rejected.

This method is not foolproof. If traffic can be decrypted then a valid MAC address could be determined and used to access the network.

Separate VLAN/DMZ for Wireless LAN

Most medium to large networks utilise virtual LAN’s (VLAN’s) to separate IP networks into logical groupings for security, performance and management reasons. Most good quality layer 3 switches provide VLAN’s that allow you to define firewall rules or access lists to restrict what resources on the network that wireless networks clients can access.

For more protection, the wireless network can be placed behind a fully featured stateful firewall rather than a VLAN alone.

WPA Wireless Clients Sample Implementation

This section shows an example of implementing WPA Enterprise with PEAP. Three components are required.

• The supplicant (the WIFI client),
• The Authenticator (the Access Point) and;
• An Authentication Server (A Radius server such as Microsoft IAS Server).

In this example the Access Point is a Cisco Aironet 1310 Series Outdoor Access Point/Bridge1 running IOS 12.3(7)JA.

In addition, since we are using PEAP, we need a suitable certificate server and we have used Microsoft Certificate Services.

Since we are using Microsoft IAS for Radius, which integrates with Active Directory, we have created 2 new Active Directory groups called "Wireless Users" and "Wireless Computers" respectively. These groups will be used to determine what computers and users are allowed access to the WIFI network. This ensures a high level of granularity for IT staff to manage access.

Configuring IAS (RADIUS)

IAS is a free component that is supplied with Win2k/Win2003 but is not installed by default. It can be installed via the Control Panel->Add/Remove Programs/Windows Components applet.

Both IAS and the authenticating access point need to be configured to perform Radius authentication. Firstly, you need to register IAS in Active Directory, so that IAS policies can be used on Active Directory users and computers to govern access.

In the left hand pane of the IAS management console, select Internet Authentication Service (Local) and right click to select the Register Server in Active Directory option.

If the server is already registered in active directory, this message will be shown

Adding the Access Point as a Radius Client

In IAS, you create a new RADIUS client and enter a shared secret.

Adding the IAS Server as a Radius Server on the Access Point

On the access point, the detail of the IAS server is entered.

Next we need to configure the Access Point to accept WPA clients and authenticate them to Radius. To do this, we first set encryption to WPA/TKIP.

Next we set the SSID to use Open Authentication and Network/EAP authentication. Network EAP is the better of the two, but Cisco have noted that ‘Some non-Cisco Aironet client adapters do not perform 802.1x authentication to the access point/bridge unless you configure Open authentication with EAP.’

We have already configured the RADIUS server, so we just use the default list. Under Key Management, we set it to mandatory and choose WPA. Accounting can also be enabled, and log on and log off events will be sent to the RADIUS server.

Configuring a Digital Certificate for the IAS Server.

If the server running IAS is already part of Active Directory, and there is a Certificate Server on the Active Directory Domain, the IAS server will already have a computer certificate installed. This can be verified using the Certificate Services MMC Span-In.

Open the Local Computer certificate store, choose Personal/Certificates and you should find a certificate matching the name of this computer. For example:

If you do not have a computer certificate, you need to obtain one via a certificate authority. This is not covered in this document.

Configuring IAS Wireless Policy

At this point we now need to configure the policy on the Radius server so that when a wireless device connects, the conditions that allow it to join the network are defined. The policies set will be to ensure the computer and user id supplied are members of the "Wireless Users" and "Wireless Computers" active directory groups.

User Access

The first policy to be created is for the user accounts that will be connecting to the wireless network. The wireless connection type will be defined and then the appropriate active directory security group will be specified.

In IAS Management console, create a new Remote Access Policy, select "Set up a custom policy" and enter "WFIC Wireless User Access" as the Policy name.

Now we add the conditions required for this remote access policy

Select Client-IP-Address, Add the IP address of the Wireless Access Point and click Ok

Select Windows-Groups and then click Add. Enter the wireless user groups "Wireless Users" and "Wireless Computers" as described earlier and click OK

Now we have completed the conditions for using this wireless policy. Click Next and grant remote access permission.

Now we have to specify the PEAP settings. Click Profile

Select the Advanced tab, highlight the Framed-Protocol (RADIUS Standard PPP) attribute and click Remove

Select the Encryption tab

Select the Encryption tab and remove all tick boxes, leaving only the Strongest Encryption (MPPE 128-bit) option ticked

Select the Authentication tab

Select the Authentication tab and click EAP Methods

Click Add to add an EAP type

Select PEAP and click OK. Now Click Edit to modify the PEAP type

Ensure that the certificate is the correct and ensure that MSCHAP is set as the EAP type

Back on the Authentication tab remove all other tick box options then click OK

You may see a message asking if you would like to view the online help to follow and configure dial-in settings

Click No and Next to continue. The wizard will now tell you the profile is complete and summarise the conditions and profile settings.

Computer Policy

Another policy can optionally be created for Domain Computers. The policy steps are identical to the steps outlined in the user policy except during the setting of policy conditions, you use the Active Directory group that the wireless computers are a member of.

Granting Active Directory Rights

The last step is to ensure that the user and computer accounts in Active Directory have rights to ‘dial in’ – which is Microsoft’s term for using Radius Authentication.

A user account needs "Control Access through Remote Access Policy" set.

A computer account needs the same setting.

Wireless Client Configuration Settings

This section only deals with the configuration of Windows XP Clients

Digital Certificate

Using PEAP, the WIFI clients do not need to use a digital certificate. Only the server uses one to identify itself to the client. However, the clients need to trust the certificate authority (CA) that issued the server certificate. To do this, the certificate of the CA needs to be added to the trusted certificate list on the client.

If the computer is a member of the Active Directory Domain, the CA certificate is likely already installed. This can be checked using the same steps defined in the section "Configuring a Digital Certificate for the IAS Server" except you go to the "Trusted Root Certification Authorities" store to find the CA certificate.

If the certificate is not installed and if the CA certificate Using Microsoft Certificate Services, it is easy to obtain and install it. The certificate can be installed by double clicking on the certificate file stored on a shared folder, floppy or USB device. Alternatively, the client can request the CA certificate from the CA website.

Wireless Networking Configuration

In the properties of the wireless network adapter, add a new wireless network, specifying WPA and TKIP

On the authentication tab, choose PEAP as the EAP type

Click Properties and tick ‘validate server certificate’ and ensure the CA that issued the server certificate is selected.

If the certificate authority is not listed above, than this machine does not trust it and it needs to be added to the trust list as per section titled "Digital Certificate"

Ensure ERP-MSCHAPv2 is selected at the ‘select authentication method’ list and click ‘properties’.

This dialog box allows you to select if XP will attempt user authentication based on the currently logged in user, or prompt for credentials to be entered. If this workstation is a member of the domain, and the user logs into it via a domain account, this should be ticked.

Click OK and OK again to return to the Wireless network properties screen.

If a computer authentication policy has been chosen as per section entitled "Computer Policy", tick the "Authenticate as computer when computer information is available".

At this point the wireless client should be able to connect to the wireless network if the user and computer credentials are valid. The event logs on the IAS server now will show detailed records of all wireless network authentication attempts including username, date/time, IAS policy used and whether access was granted or denied.

Below is a sample IAS log entry from an access point using the IP address 192.168.100.26 and an IAS policy called "AP Test".

Event Type:    Information

Event Source:    IAS

Event Category:    None

Event ID:    1

Date:        20/01/2006

Time:        4:45:07 PM

User:        N/A

Computer:    MYSERVER

Description:

User DOMAIN\some guy was granted access.

Fully-Qualified-User-Name = domain.net/MY Company/DOMAIN/Some Guy

NAS-IP-Address = 192.168.100.26

NAS-Identifier = bridge

Client-Friendly-Name = AP Test

Client-IP-Address = 192.168.100.26

Calling-Station-Identifier = 0013.ff10.427c

NAS-Port-Type = Wireless – IEEE 802.11

NAS-Port = 282

Proxy-Policy-Name = Default Connection

Authentication-Provider = Windows

Authentication-Server = <undetermined>

Policy-Name = Test – Wireless

Authentication-Type = PEAP

EAP-Type = Secured password (EAP-MSCHAP v2)

                      

No Tags