As I write this series, it is getting less and less about Cisco and more and more about SharePoint. This article is definitely developer centric, but since Cisco guys tend to be interested in the guts of the detail, I decided to keep going .

If you read my articles I tend to take the piss out of IT role stereotypes just to make it more entertaining reading. Sales guys and IT Managers tend to cop it the most, but I also like to have a dig at the expense of the nerds too. Cisco nerds on the whole are a great bunch, but I have to say, the scariest nerd I have ever met drank Cisco kool-aid in jumbo size!

If you have gotten to this article after reading the first two and you are scoffing at my audacity to suggest you TFTP your configs into SharePoint, chances are most people think you’re scary! If you are hitting this series of articles for the first time, go back and read part 1 and part 2 before being scary!

Seriously now, I thought that this would be a 2 part set of articles, but I got all bogged down in the methods of getting files into SharePoint. The WEBDAV based methods described in the previous article is easy to do, but ultimately is not the recommended method. So now, we will look at the ‘proper’ ways to do it and see if they are worth the effort. They work okay, but are more complex and I’m not convinced that the governance issues are necessarily worth it for many readers.

Degree of difficulty for this article is varied.

CleverWorkArounds Coffee requirement rating (for an application developer):  

CleverWorkArounds Coffee requirement rating (for a non developer):    

How can we we *not* use WEBDAV?

If we are using methods other than WEBDAV, then how are our existing applications supposed to work? I mean, all they know is the windows networking API. So, unless the other methods are implemented as a redirector (as described in more detail in the “WEBDAV Interlude” section of my first article), how can they use any other API?

The answer is, they can’t! So we are leaving the world of freeware and cheap TFTP servers and writing our own one!

“What? I’m no coder”, you say? That’s okay, nor am I… but stick with me anyway.

Okay, how about we modify someone else’s open source TFTP Server to work with SharePoint via non WEBDAV methods?

Fortunately for us, such a TFTP server does exist. In addition to being conveniently modifiable (got to love open source), it is multithreaded, very well written and easy to understand and extend.  Kudos to the author. If you write a few scripts here and there, it should be enough for this article.

So let’s take a closer look. If you already know visual studio you can skip the next section and move to the section entitled “Examining the Code”. Otherwise let’s keep going getting this TFTP Server compiled!

Getting TFTPUTIL compiled

Head on over to http://sourceforge.net/projects/tftputil and download the latest release. Crank out Visual Studio 2005 and take a look at it.

This server is implemented as a class, so you have to write your own little program to make use of this class by creating a TFTP server object. You do by adding a new project to your Visual Studio workspace. But before we do that, how about we deal with a pre-requisite component.

The TFTPUtil class makes use of a library of useful functions called NSpring. You will want to download this. Grab the nspring.dll, and then tell Visual Studio how to find it by choose “Add Reference” from the “Project” menu, . At the resulting dialog box, go to the “Browse” tab and navigate to where you downloaded nspring.dll. (Best to put into C:\WINDOWS\SYSTEM32 or use the same folder where the TFTPutil code lives).

Right, we are all set! You could compile and build TFTPUtil.DLL right now, but it would not do you a lot of good because it’s a class and does not run on its own. It needs to be *invoked*. This means we need to create a little application that creates an *instance* of the TFTPUtil class. Right click on the project and choose “Add New project” as shown below.

For our purposes, we create a new “Console Application” project and call it myTFTPServer or something meaningful to you. This will be a small EXE file that uses the TFTPUtil class to create a TFTP server.

Now, right click on the newly created project and choose “Set up Startup Project”

Next step, we need to tell this new console application project how to find the TFTPUtil class. From the Project menu, choose “Add Reference”. At the resulting dialog box, go to the “Project” tab and you will see the open TFTPUtil project.

Right, that should be it! Your Visual studio should look something similar to this

Creating a basic TFTP server

This is a well written TFTP server. Everything is very nicely modular and easy to understand without getting into too much complexity.

So first up, here is the 5 lines of code of your console application to get the TFTP server working. Add this to the code section of the static void Main(string[] args) method.

using TFTPUtil;

TFTPServer myTFTP = new TFTPServer();
myTFTP.Path= "C:\TEMP";
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.StartListener(); 

Those 5 lines create a new TFTP server object, set the destination folder, allow it to receive files, overwrite received files and start the TFTP server. If you now compile and build this solution you will end up with the files TFTPutil.DLL and myTFTPServer.EXE.

You should be able to test it now. (By default, files will be uploaded to the folder the EXE is run from unless you specify a path).

Examining the code

Now let’s look at the area of the TFTP server where we need to add some code. It is in TFTPServerProcess.css in a method called “ProcessDataGram”

If you examine this method, you will see the following code (just search for it):

AddMsg(Level.Info, "Successfully received file " + State.Filename + " (" + State.Filesize.ToString() + " bytes) from " + State.RemoteIPAddress + " in " + TotalTime.ToString() + " seconds (" + (Rate).ToString() + " bytes/sec).");
//This is the last block so go ahead and setup to delete state
State.Close(); 

We will make some fairly basic modifications to the server in this section, so that the file is uploaded to SharePoint.

There are several methods that can be used to get files into SharePoint. We will do this first by using the most obvious way: The SharePoint SDK.

Modification example #1 – The SharePoint SDK

The SharePoint SDK is definitely the easiest way to upload a file, but there are several caveats:

• The TFTP server MUST run on a server that is a member of the SharePoint farm
• The TFTP service account MUST have administrative privileges on the server and SQL access

That second requirement in particular just plain sucks. After going to the trouble in part 1 to set up low privileged account “tftp service” with “contribute” access to a document library, we now find that we need to grant the account significantly more privileges than before. My security nazi Cisco readers will read that and say “no way will I do that”, and I would concur. But fortunately for my readers (and unfortunately for me) I only discovered that caveat AFTER I had gotten it working. So I will show you the code changes anyway.

So, first up I will tell you what we will do. The aforementioned ProcessDataGram uses a class called State that is used to write the file to the file system. Then later the method Close() closes the file that has been uploaded.

So what we will do, is upload the file to SharePoint just after State.Close(). This means the file is still downloaded to the local file system and then copied to SharePoint. Therefore, the local file system is now simply a temporary place for the file to be stored, prior to it be uploaded to SharePoint.

To do this, we will change how TFTPUTIL handles the PATH property. Recall in my basic TFTP console application example code I could set the path via:

myTFTP.Path = "C:\TEMP";

So let’s change this behaviour so that if Path contains the string “http://”, we assume it is a SharePoint path, rather than a file system path. Therefore we set the internal file path to %TEMP% so we can still capture the file, but then we will upload it to the http:// specified SharePoint site.

Path is a property of the TFTPUtil class that is defined in TFTPServer.cs. We will make a minor change to the Path property, because it performs a check that will always fail when a SharePoint URL is supplied.

public string Path
{
    get
    {
        return FullPath;
    }
    set
    {
        if (value.Contains("http://")) {
            FullPath = value;
        } else
        {
        if (System.IO.Directory.Exists(value))
            FullPath = value;
        }
    }
} 

Now we turn our attention to the TFTPServerProcess.cs file where we have to modify the TFTPServerProcess class. We will add two more properties to the TFTPServerProcess class. Find the declaration for FullPath and then add the two lines in bold below it.

private string FullPath = System.IO.Directory.GetCurrentDirectory();
private string TempPath = System.IO.Directory.GetCurrentDirectory();
private bool IsSharePoint = false; // If this is a sharepoint site 

Also while we are here, add a reference to the SharePoint dll.

using Microsoft.SharePoint; 

You will find the Microsoft.SharePoint.DLL in C:\Program Files\Common Files\Microsoft Shared\web server extensions on most installations.

Next, we need to add some basic logic to the code where the TFTPServerProcess class is initialised. It is an overloaded class so it is declared twice. To keep it simple, I have added the code to each declaration. If this is a Sharepoint path, it copies the supplied path to a variable called TempPath, and then puts the path of %TEMP% into the FullPath. This is deliberately done so that minimal code changes elsewhere are required (ie the rest of the class expects a variable called FullPath to point to a filesystem, not SharePoint).

Thus, it looks like this (bold):

Declaration 1:

public TFTPServerProcess() 

if (FullPath.Contains("http://"))
{
    IsSharePoint = true;
    TempPath = FullPath;
    FullPath = Path.GetTempPath();
}

Declaration 2:

public TFTPServerProcess(string FilePath,
    Level LoggingLevel,
    Level DisplayLevel,
    bool AllowRRQ,
    bool AllowWRQ,
    bool AllowWRQOverwrite,
    bool AllowTFTPOptions,
    bool CheckReadWriteState,
    int ResendInterval,
    int TimeoutInterval,
    Logger logger,
    IPAddress srvaddr)
{ 

    if (FilePath.Contains("http://"))
    {
        IsSharePoint = true;
        TempPath = FilePath;
        FullPath = Path.GetTempPath();
    }
    else
    {
        FullPath = FilePath;
    }

Finally, we add some code just after the State.Close() method as shown earlier. This takes the downloaded file from %TEMP%, uploads it to SharePoint and then deletes the file from %TEMP%.

State.Close();
if (IsSharePoint)
{
    // Now we add this newly downloaded file to the document library
    try
    {
        SPSite DestSite = new SPSite(TempPath);
        SPWeb DestWeb = DestSite.OpenWeb();
        DestWeb.AllowUnsafeUpdates = true;
        SPFileCollection files = DestWeb.Files;
        FileStream destFile = File.OpenRead(FullPath + @"/" + State.Filename);
        string destFileURL = TempPath + "/" + State.Filename;
        Hashtable MetaDataTable = new Hashtable();
        SPFile destSPFile = files.Add(destFileURL, destFile, MetaDataTable, true);
        destFile.Close();
        // clean up temp file
        File.Delete(FullPath + @"/" + State.Filename);
    }
    catch (Exception ex)
    {
        AddMsg(Level.Verbose, "Problem opening SharePoint site" + TempPath);
        AddMsg(Level.Debug, "Exception: " + ex.Message + "\n" + ex.StackTrace);
    }
}
//This is the last block so go ahead and setup to delete state
StopListener(); 

Note the HashTable MetaDataTable line. Even though we are not setting any metadata values, I have left in HashTable code because it illustrates how this method allows you to code in the setting of metadata columns in a document library. For example, imagine that the SharePoint document library has a column called “DeviceType”. Your code could say, write the string “Cisco” or “Linksys” to the DeviceType column when the file is uploaded.

So let’s try it out shall we?

Modify your console application to fix the path variable to a SharePoint doc library and then build the solution. 

TFTPServer myTFTP = new TFTPServer();
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.Path = "http://tidemo/tftp/backups";
myTFTP.StartListener(); 

Test the solution and you get a fairly unexciting console window.

Now we attempt an upload.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip it all working swimmingly]

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Also, this time around, none of the locking/access errors seen with the WEBDAV based methods – WOHOO!. Running this as a service also works fine, using the same freeware utility I used in the last post (ServiceInstaller ). Just compile the solution and then use ServiceInstaller as I described for TFTPD32 in the previous post.

Remember what I said before though, the “tftp service” account MUST have the following privileges on the server:

• The user is a server farm administrator.
• The user has Read and Write permissions on the content database.
• The user is a site collection administrator.
• The user has permissions to access the Windows SharePoint Services site or the SharePoint Server 2007 site through which the code iterates.

Read more at KB Article 935751, but the fundamental issue is that we are not running this code as the IIS process, but rather a separate user account. “If code is executed from a custom Windows-based application or from a custom console application, the code path does not use the Web context that is hosted by the Microsoft Internet Information Services (IIS) application pool. Therefore, specific user permissions are required in SharePoint Server 2007 and in the back-end data source”

Conclusion

The requirement for administrator level privileges sucks. It is not a showstopper, but it sucks nonetheless. So the next post, we will turn our attention to the Webservices methods of uploading data and see if they are any better. Then (finally!) we will get onto more interesting topics like making use of workflow and integrating it with other useful information.

If you are wondering whether you can download my changes and test for yourself, not yet . I’m going to contact the author of the TFTPUtil project first, and I have to perform more debugging . So right now you will have to do it yourself.

Thanks for reading.

Paul

No Tags

Here I am back again, illustrating some of the interesting possibilities that SharePoint offers for Cisco people.

To recap my last post, I showed you a little perl script I wrote to get an IOS router or switch to dump its current configuration to a TFTP server. I then used one of several freeware TFTP servers to show how you can have a TFTP server save the captured file into a version enabled document library.

I then hit a snag in relation to using a Windows Service to do this task. In this article we will delve into this issue in more detail. In addition, I ended up delving much deeper than I intended. So, like my branding series, this is going to turn into a multi-part series too, covering some application development, configuration, security and governance issues. How many parts it will end up being is anybody’s guess!

This is a technically oriented series of articles for the most part, so for you people who like the governance and finance stuff, you may not get too much out of this one. Although this article (part 2) focuses on my issues and observations with the Windows WEBDAV client, if you are one of these people who have ‘special’ feelings when you see those pretty blue Cisco boxes like the image above, then you may find some useful content here.

SharePoint developers and architects may also find this of interest.

Now I previously mentioned that I tried three TFTP servers in writing this article and had some problems. They were:

• WinAgents TFTP (Commercial)
• SolarWinds TFTP Server (Freeware – Registration required)
• TFTPD32 TFTP Server (Freeware)

And let’s not forget good old PUMPKIN, just to make one of my CCIE friends, who read the first post and suggested I use it as an alternative, happy (although he forgets that years ago I was the one who told him about Pumpkin in the first place )

Let’s review their capabilities and cover the issues encountered with each. As we go, we will get to see the uglier side of WEBDAV.

WinAgents TFTP

I’ve used this product several times in the past. At the time it was pretty much the only product that would natively run as a Windows service. It is not free, but buying it won’t exactly break the bank. It has a nice GUI, good logging and the ability to set up virtual folders to different underlying locations. Below is the main configuration screen and the screen where multiple locations can be configured.

You would think that the above ability to virtualise folders would make this product an obvious choice for front-ending to SharePoint, because it could co-exist with native filesystem, DFS and 3rd party SMB based resources like those provided by samba.

You would be right, but the latest version is let down by one stupid design flaw. When you send a file to a WinAgents TFTP server that already exists, it will delete the original file and then copy the new file to the virtual folder. It may on the surface seem no different to simply overwriting a file – but there is a big difference in a SharePoint site.

All files in a SharePoint document library have a unique ID. It is a built-in column and can be made visible by modifying a document library view as shown below.

By deleting the file in SharePoint, that file is now put into the recycle bin. When you upload a file of the same name, SharePoint treats it as a new file and allocates it a new ID. Therefore, there is no version history at all. Each upload is deleting the previous version and uploading a new file.

This obviously blows away any potential to use WinAgents TFTP as a SharePoint integrated TFTP server .

CleverWorkarounds Rating: They probably thought it was a good idea at the time.

Solarwinds

So instead, let’s now turn our attention to the SolarWinds TFTP product. This one is free and also very easy to use. Like the WinAgents product, it can be run as a service out of the box, but it is not as comprehensive as its commercial competitor in terms of logging, granular security filtering and folder virtualisation.

Unlike its commercial cousin, however, it does not delete files before uploading, and therefore plays nice with SharePoint version control. That, combined with natively running as a windows service, makes it the obvious contender to be the SharePoint TFTP server of choice.

The GUI is basic yet functional. There is a status screen and a configuration screen that allows you to set the TFTP Root directory as shown below.

In the screen-shot above, you will notice that the TFTP Server root directory is \\tidemo\tftp\backups. As I explained in the first post, this is the WEBDAV UNC equivalent of the SharePoint site http://tidemo/tftp and the document library called backups.

If you recall in the first post, we created a user account called ‘tftp service’. This account has been granted access to the http://tidemo/tftp/backups library. So, we need to set the credentials of the WinAgents TFTP service to run as that user. Therefore, when files are received by TFTP clients, the server will be able to write to the library. Below is my screen capture of my configured TFTP server.

There was one other task I also had to do to test this setup. SolarWinds is a little bit naughty in that it logs to its installation folder. (‘tsk ‘tsk will never get a designed for XP gold logo). Being good practitioners of IT security, the ‘tftp service’ account is a domain user with no system privileges, and only has READ access to this folder by default. Changing configuration parameters (such as the UNC TFTP root) also would not be saved, because the TFTP server would not be able to write to its own configuration file when running as the ‘tftp service’ account..

So the permissions of the SolarWinds installation folder has been modified so that the ‘tftp service’ account has WRITE access as shown below.

So I started the service fine, and ran a test! Wohoo we have our file!!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

So I tried it again a few times and hit a snag after some successful retries…

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\TFTP 192.168.134.129 put myfile.txt
Error on server : The process cannot access the file ‘\\tidemo\tftp\backups\myfile.txt’ because it is being used by another process.
Connect request failed

This was odd. Suddenly I was unable to copy a new version of the file to TFTP. I was running many repeat tests, so perhaps there was some sort of temporary file lock causing a problem? So I waited for a bit and then was able to transfer successfully again as shown below.

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

But it wasn’t long before I got another and more severe error.

C:\TFTP 192.168.134.129 put c:\myfile.txt
Error on server : The network path was not found.

Connect request failed

The TFTP server log showed this error and ominous stack trace…

2008-01-23 21:49:53,102 [6] WARN  TFTPServer.Service.TransmissionPut – Exception caught while trying to open file for reading.
System.IO.IOException: The network path was not found.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access)
   at TFTPServer.Service.TransmissionPut.Begin(RequestPacket firstPacket)

Network path not found? What the hell was going on here? It worked a minute ago!

Eventlogs came up blank, and I tried an IISRESET. Still no-go. Restarted the TFTP service and I was able to transfer again. However, it didn’t take long for the problem to re-appear.

I tried all of the suggestions on Mark Muller’s blog article with no success.

I found two sites of interest and one seemed close to this issue. The first was here, where a developer was having the identical problem, and a Microsoft employee suggested not to rely on WEBDAV “especially when using services”. The second one was a confidence building quote from Microsoft themselves. They made an admission in their WEBDAV Whitepaper regarding the Web Client service. “The Web Client Service in Windows XP and Windows Server 2003 does not function correctly if you stop it then start it again without restarting the computer. Although a bug has been opened on this problem, the changes required to fix it are too large for a hotfix or a service pack and have therefore been delayed until a future major release of Windows”

Are you freakin SERIOUS!?!

Anyway, I decided that before I go off on a Microsoft bashing rant, I’d better double check that this problem was with SolarWinds TFTP or the obviously rock solid and stable WEBDAV client.

CleverWorkAround Rating (SolarWinds TFTP) – Judgement reserved till later

TFTPD32 TFTP Server

TFTPD32 is more than just a TFTP server. It also provides DHCP, Syslog and SNTP services as part of the deal. Unlike the previous two TFTP servers, it does not run as a windows service out of the box, but it can be easily made to.

Cisco have also decided that this is their recommended TFTP server as well, not a bad endorsement, eh!

If we focus on the TFTP server functionality only, it is basically the same as SolarWinds without the windows service bit. So WinAgents still wins for overall ease of use and granularity but as discussed it was a waste of time with SharePoint.

Once again there is a main status screen, showing you current activity, as well as a configuration screen where you set the TFTP server root UNC path.

So given that we had all of the WEBDAV stability problems with SolarWinds, this TFTP server allows us to test two scenarios. Since it can be run as standalone, we can run it interactively as our ‘tftp service’ account and test it. Then, we can use a 3rd party tool to make it run as a Windows Service, and compare notes.

So first up, let’s run it interactively as rtftp service’ account. This is achieved by RIGHT clicking on the TFTPD32.EXE file and choosing RUN-AS from the menu options. When prompted, enter the credentials of the ‘tftp service’ account.

If you want absolute assurance that TFTPD32.EXE is being run by the ‘tftp user’ then look for the process in “task manager” and check the “User Name” column.

So, here we go!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Now occasionally I received the error below, but unlike when using SolarWinds, running as a windows service using ‘ttp service’ user credentials, it would always happen once and subsequent attempts would be fine.

C:\TFTP 192.168.134.129 put myfile.txt
Error on server : Access violation
Connect request failed

Just to prove that it was much more reliable, we now look at the version history in SharePoint for the file myfile.txt

CleverWorkArounds Rating: Not flawless, but reliable

So we have ascertained that running TFTPD32 works pretty well when run interactively as ‘tftp service’ . Now we have to run TFTPD32 as a service and compare

TFTPD32 as a windows service

There are many utilities out there to run a regular EXE file as a Windows Service. One of the most popular ones is SVRANY from the Windows 200x resource kit. However I used the freeware ServiceInstaller because it had a nice GUI

Using it is mind numbingly simple. Run the application and fill in the form as shown below and click the ‘Install’ button. Once installed, you can then change the service credentials to ‘tftp service’ the same way we did it for SolarWinds.

Start the service, and check in Task manager that TFTPD32.EXE is running. (When running as a service you will not see the status screen).

The moment of Truth!

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many successful reattempts]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Like when running interactively, I still received the occasional  ‘Access Violation’ error but always recovered. Again to prove that I ran lots of TFTP commands, the version history is now up to 204 .

CleverWorkArounds Ratings (TFTPD32 as a service) – The best of the WEBDAV TFTP bunch

CleverWorkArounds Ratings (SolarWInds) – Looks like it might be part your fault after all!

CleverWorkArounds Ratings (WEBDAV) – It’s definitely your fault!

Conclusion

So after these tests, it seems that running TFTPD32 as a Windows Service is no less reliable than running it interactively. Perhaps then there is an issue with the way that SolarWinds use the windows networking API’s to write files. SolarWinds is a .NET application and TFTPD32 isn’t. Therefore, they may well be using different API’s under the hood.

I wasn’t satisfied with all of this though. Although we have achieved our goal of TFTP to a version controlled document library, the whole unreliable WEBDAV thing left me with a bad taste and I decided to go down a different path. What more could I do, you ask? How about write my own TFTP server?

The next 2 posts in this series will cover getting TFTP files into SharePoint via *non* WEBDAV methods. We will examine their reliability and examine a little of SharePoint web services and the SDK.

The fifth article will leave the issues of getting the files *into* SharePoint, and instead focus on what we can do with them once they are there! (Workflows, metadata and the like).

The final article (I suspect it will be the final anyway but i won’t know ’til I write the next two), will tie it all together in terms of governance. Thus, it will be a little similar to the branding series where the last article was largely non technical in focus and audience.

I hope you’ve enjoyed this little journey and didn’t find it too pointless

No Tags

Cisco nerds! This series is just for you! I know that you think you’re way too cool for collaborative portals, especially a Microsoft one at that. Instead you are more interested in delving into the IOS command line, to perform arcane arts such as debugging that OSPF route redistribution into BGP or getting off on planning and implementing a large scale DMVPN solution. Maybe you’re into QOS and VOIP and simply dig all of those DSCP-COS mappings, class and policy maps and the like.

Although packets, cells and frames are your world, *nix is cool, Nagios is your idea of a portal and anything remotely connected to Microsoft fills you with contempt and is beneath you right?

Well if this is you, I do understand your point of view because I was you once, but after some therapy, I’m now out of rehab and doing just fine!

Having Cisco/general networking expertise will help you with this article, so depending on who you are, the amount of caffeine required to follow this will vary:

CleverWorkArounds Coffee requirement rating (for a CCNP or CCIE):  

CleverWorkArounds Coffee requirement rating (for a non Cisco person or CCNA ):  

Okay Cisco gurus, if there is anything about SharePoint that you guys need to know about, it is the power of the document library.

Unlike a regular file system, a SharePoint document library has a powerful version control mechanism that can make your life easier, as well as play nice with the rest of your colleagues (whom you despise because they don’t understand or appreciate you ).

So I am first going to show you an example of how you can leverage SharePoint to do something quite useful with Cisco gear, and then in the next post of this series I will expand further into general IT infrastructure admin benefits.

The Cisco bit..

(Non Cisco people, your eyes will glaze over – but try anyway!)

Back around 2000/2001 when I was a full time network manager, I wrote a perl script that used SNMP to instruct a router to dump its configuration to a TFTP server. I had a cron job that ran this script daily, and it would tell all of our switches and routers to write their current configuration to TFTP. It was nothing special – many Cisco network professionals do the same thing.

(Before you ask, I *know* there are tools and new methods to do this nowadays, but back then the choices were more limited)

Anyway, here is a snippet of the script in question. I know it’s not great but it did the job…

use Net::SNMP;

($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localtime;
@Days = (“Sunday”, “Monday”, “Tuesday”, “Wednesday”, “Thursday”, “Friday”, “Saturday”);
$day = $Days[$wday];

# Now call the procedure to do the deed!
&backup_configs(“192.168.16.1″,”RWSNMPcommunity”);
&backup_configs(“192.168.16.2″,”RWSNMPcommunity”);

The above code simply grabbed the date and then called the main function, passing the IP of the router and the SNMP community string for Read/Write access. The routine below, used SNMP SET commands to tell the router the location of the TFTP server, the name of the configuration file being written, and then told the device to execute the request.

sub backup_configs {
    ($ipaddr, $cs) = @_;
    ($s, $error) = Net::SNMP->session( -hostname  => $ipaddr,
                                -community => $cs,
                -debug => ’1′ );

    $hnh = $s->get_request(“.1.3.6.1.2.1.1.5.0″);
    $hostname = $hnh->{“.1.3.6.1.2.1.1.5.0″};

    $filename = “\\router-configs\\” . $day . “\\” . $hostname . “.cfg”;
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.2.667″, INTEGER, 1);
    # Set the source to startup-config
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.3.667″, INTEGER, 4);
    # Set the destination to tftp
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.4.667″, INTEGER, 1);
    # Set TFTP IP Address
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.5.667″, IPADDRESS,”192.168.18.5″);
    # Set destination file name
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1x.1.6.667″, OCTET_STRING,”$filename”);
    # Set it off!
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.14.667″, INTEGER, 1);
    $done = false;
    $counter = 0;
    while ($done eq false) {
        $check = $s->get_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.10.667″);
        if ($check->{“.1.3.6.1.4.1.9.9.96.1.1.1.1.10.667″} == 3) {
            $done = true;
        }
        $counter++;
        if ($counter > 500) {
            $done = true;
        }
    }
    if ($counter > 500) {
        print “$date:A problem occured – it looks like the attempt failed\n”;
    }
    # Clear the config!\n”;
    $s->set_request(“.1.3.6.1.4.1.9.9.96.1.1.1.1.14.667″, INTEGER, 6);
    #close the snmp session
    $s->close;

}

Now, I know there is a lot of ugly SNMP stuff in my above code and many readers are thinking, WTF!? You can safely ignore most of the code. However there is one bit that I do want to bring your attention to.

    $hostname = $hnh->{“.1.3.6.1.2.1.1.5.0″};

    $filename = “\\router-configs\\” . $day . “\\” . $hostname . “.cfg”;

Looking closely at this, you can see that I am setting the filename based around the name of the Cisco device (retrieved from SNMP) and the day of the week the script is run ($day). This is because when I backed up the configuration of the router, I wanted to keep multiple versions. For simplicity, I opted to keep 7 days worth of configuration dumps. So by using the day of the week in the file path, I ended up with config backups like this.

• router-configs/monday/mycorerouter.cfg
• router-configs/monday/myedgerouter.cfg
• router-configs/tuesday/mycorerouter.cfg
• router-configs/tuesday/myedgerouter.cfg

So, provided the path router-configs/<day of the week> exists on my TFTP box, I end up with a single file for each cisco device in each day of the week directory.

Neat, eh?

Yeah it’s neat, so why integrate TFTP with SharePoint?

Version control and workflow, my skeptical Cisco friends, that’s why! The easiest way to explain is to dive right into an example. So let’s do that right now!

There are a few TFTP servers out there for Windows (yeah we are using Windows – get over it). I used to use WinAgents TFTP server. Unfortunately though in writing this article, the most recent version of WinAgents TFTP does not work with SharePoint the way I want it to (if you run an older version it is fine). I also tried using SolarWinds freeware (registration required) TFTP server as it runs as a windows service. Unfortunately, I had some stability problems with WEBDAV when running as a service (not SolarWinds fault) which I will explain in the next post, so I ended up using TFTPD32 with a few modifications. (TFTPD32 is recommended by Cisco anyway).

What we will do is the relatively simple process of telling the TFTP server to drop files into a SharePoint document library via WEBDAV.

For reference, the TFTP server does not have to be on the same server as SharePoint when using the WEBDAV method.

The TFTPD32 Server Installation is straightforward. Once installed however, there are three additional steps that you need to perform:

• Create or nominate a user account to run the TFTP Service.
• Modify TFTP32 to run as a WIndows Service – using the credentials of a user account that has permissions to the document library that the TFTP server will dump the files to.
• Set the TFTP root directory to the UNC path of the SharePoint document library

I am assuming that you already have a SharePoint or WSS server. On the SharePoint side, we need to create a site and a document library to hold our configuration backups, grant the TFTP server permission to it and then enable version control on that document library.

So in my example, the user account that is going to upload files to SharePoint is called “tftp service“. I will refer to this account frequently in the rest of this post.

Also in this example, the SharePoint site is http://tidemo/tftp and the document library is called “backups“

http://tidemo/tftp/backups

Configuring SharePoint

(SharePoint people, I am covering the basics here – skip if you know it already).

In SharePoint, create a new, blank site to host the TFTP server files. Being a bastion of originality, in the example below I have named the site TFTP.

Now that the site has been created, we create a document library to store the files for the TFTP server. I have called it “Backups” and chosen not to use a default document template.

The next step, having created this document library, is to turn on version control. This is performed by choosing “document library settings” from the “Actions” menu.

In the version settings screen, choose to create major versions.

Finally, we need to grant our user “tftp service” write access to this new document library. The simplest way to do this is add “tftp service” to the members’ group of the SharePoint site. This will grant them ‘contribute’ access to the “Backups” document library. (This permission setting is just for demonstration purposes. Adjust permissions according to your specific requirements)

Right! Our doc library is ready! If at this point you open the document library via Windows Explorer as shown below, you will see that the UNC equivalent path for this example document library is \\TIDEMO\TFTP\BACKUPS

If you do not see your standard windows explorer window here and a UNC path, stop here! We need to troubleshoot the windows WEBDAV service. It might pay for you to skip to the “WEBDAV interlude section coming up)

One thing that people often overlook with SharePoint is that document libraries have this “windows explorer” view. What is so significant about this, you ask? It is the fact that you can reach any SharePoint document library via a UNC path. You can even map a drive to a SharePoint library too. Thus the following document library:

http://tidemo/tftp/backups

is accessible via

\\tidemo\tftp\backups

As a result of this functionality, pretty much any application that deals with UNC and mapped drives (such as ROBOCOPY) can happily copy files from your PC to a SharePoint library (with a large caveat I will explain later).

For educational purposes, let’s look at how windows explorer view is done. (If you are not interested, skip to the next heading where we configure SolarWinds TFTP.)

How is this done? A WEBDAV interlude

Now SharePoint does not actually provide support for SMB which is what ‘regular’ shared file systems use. The protocol that SharePoint supports is WEBDAV which stands for “Web-based Distributed Authoring and Versioning”. So, how is it that we can use Windows explorer, mapped drives and UNC paths to get to SharePoint files?

In Windows XP and above, Microsoft supplied a new network provider called the Windows WebDav client. (Yes, XP and above – Windows 2000 is not going to work this way for you). Also, Samba people who are reading this and thinking “ooh I can use smbfs to mount a SharePoint library to *nix”, sorry it ain’t gonna work because it is not SMB.

By creating the Web Client Service as one of the built-in network providers, it is automatically queried whenever you make connection attempt to a network resource (i.e. opening Windows explorer and navigating to a mapped drive or UNC path). You can see the provider by following these steps:

• On the Start menu, right-click My Network Places and then click Properties.
• On the top menu bar, click Advanced and then click Advanced Settings.
• Click the Provider Order tab.

You will see a dialog box with your provider order listed. Note that the Web Client Network (WebDAV) is last in the list by default.

Microsoft have a detailed whitepaper on how Windows Explorer view is implemented, and it is well worth a read. (I’ve previously blogged on quirks with it before.)

TFTP to SharePoint

So, now we know that on XP and above, any application that uses the windows networking API (interpret that as refer to a mapped drive or UNC path), can get data into and out of SharePoint. So let’s run the TFTPD32 and set it to use our document library.

Configuring TFTPD32

First up, grab TFTP32 and install it onto your chosen server. After successful installation run TFTP32.EXE from the installation directory.

Click OPTIONS and now set the path to the Windows explorer view path (UNC) of your site and document library

That is pretty much it! Now let’s test it!

Unfortunately, I don’t have any spare Cisco gear handy as Uncle Ebay helped me dispose of them once I stopped doing networking. So instead, I will use the built in TFTP.EXE utility built into Windows. This proves that a file can be saved to SharePoint via TFTP.

It works like this.

TFTP <hostname or IP> GET|PUT <filename>

eg

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

The output of the above command shows “transfer successful”. So let’s go and have a peek in our document library!

Sweeeet, we have a file called “myfile”.

So let’s go back to TFTP and re-run the command a few times.

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip many re-executes]

C:\>TFTP 192.168.134.129 PUT myfile.txt

Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Now let’s take another gander in the document library. Still the one file there. However, let’s now examine the version history for the file and see if there are multiple versions?

Sweeet, we now see that there are 7 versions stored for this file!

Conclusion (and the catch…)

So imagine that every day, you use the perl script example above to instruct your networking gear to copy their configuration to TFTP. Now you can simply use a standard file name convention (such as <hostname>-config.txt) do not need to worry about folder structure or dates within filename to ensure that multiple versions are maintained. You can go into the version history for a particular router configuration file and retrieve the one that you need.

Of course, all of your other important documents such as your network diagrams, SOE documentation, maintenance and support agreement details can also be placed into SharePoint document libraries and many other benefits are derived in addition to version history, such as search, workflow and easy offline storage.

But that is for the next post

Now for the catch … and it’s a biggie!

We of course, want to run our TFTP server as a windows service, so that the TFTP server does not have to be manually invoked.

Try as I might, I was unable to get a reliable TFTP service writing to SharePoint document library, when the TFTP server is running as a Windows Service! This majorly sucked and I was an unhappy camper!!

I have concluded that the WEBDAV client in WIn2003 is flakey. In the next post of this series, I will outline the problems that I had and how I eventually was able to come up with a semi-clever workaround to make it all work.

Bye for now!

Paul

No Tags