I had a problem this week that got resolved with something quite obvious, but I learned a lot in the process, so I am detailing it here.

Symptoms: Using MOSS 2007 SP2 and Office 2003 SP3, users complained that metadata on documents went missing. Consider the scenario below where we have a sample document library.

Our user opens the “Memo to Council re Convenor” document into Word 2003, makes a change and saves it. Note the difference. Where’s my metadata?

I asked about this on the ozmoss mailing list, and others noted the same issue. Some had applied the June 2009 cumulative update for WSS3 and the problem was solved for them. I applied this update, and it improved the situation, but did not cure the problem totally.

Upon further investigation, we were able to ascertain that the problem would only happen on certain PCs. The same user could update the same document on a different PC and it would work fine. A fiddler trace of the process allowed me to narrow down the issue and understand the sequence of events. The root cause of the issue was incorrect handling of HTML/JavaScript by Office 2003 and/or Internet Explorer. This manifests itself in an inconsistency in the display of properties in the Office “Web File Properties” dialog box (file->properties in an office 2003 app). Consider the example below.

Note that the first dialog shows that values for the metadata columns are blank or default. Now consider the same document opened on a different PC. Well what do you know – we have metadata!

Fiddler confirmed that when saving a document to SharePoint from an Office 2003 application, the same HTTP request is made to SharePoint as is done when displaying the document properties in the above example.

In both cases, A HTTP GET request is made to the owssvr.dll  (WSS RPC)  and the dialogview method is called. Therefore, the root of my problem has something to do with the fact that the data returned by this call was not being parsed properly by MSWord on one PC (as illustrated by the first of the above dialog box screenshots). When a user than saved their edits, blank or default values are being written back to SharePoint, in effect “losing” the metadata.

So let’s take a look at the HTTP call and the data returned. The call to owssvr.dll  looks like this.

GET /somesite/_vti_bin/owssvr.dll?location=My%20Committee/Agenda%20attachment%20July%2009.doc&dialogview=SaveForm HTTP/1.1

The output returned by SharePoint is a bunch of HTML and JavaScript that MSWord then renders by calling the Internet Explorers rendering framework programmatically. (I’ve included sample output at the end of the post for the hardcore nerds).

So the question now becomes, what was preventing the correct HTML rendering on one PC and not the other? This was a difficult question to answer because I was unable to find a way to debug JavaScript when the output was rendered in Office 2003 apps.

When you think about it, there can be many potential causes here. Given that Internet Explorer is effectively doing the work (WinINet), this whole process could be adversely affected by add-ins, zone settings, AD policies, virus scanning, etc. On one affected PC I disabled all all-ins to Internet Explorer and retried without success. After around half an hour of frustration, I decided to reset IE’s configuration as shown below.

After accepting the various warnings, I retried the test and it all worked! Metadata was now being properly saved. “Awesome”, I thought. I’m not quite sure what the true root cause is, but at least I know how to cure it.

Then it suddenly dawned on me that I’d been stupid. For all of my low level examination of the office 2003/RPC interaction with SharePoint, picking the brain of gurus like MCM Spence Harbar, I’d never thought to clear the temporary internet files cache. Doh!

On the next affected PC I did so, and the problem also went away instantly.

*Blush*

In my defence of my dumbness however, I had never examined the behind the scenes integration with Office 2003 and SharePoint, and it did not even twig that Internet Explorer would be involved in the picture. Given that this problem manifests itself within Office 2003 applications, it is not immediately obvious that your temporary internet files would cause an issue here – but now I know better .

And now I get it…

Now Office 2007 is not affected by this because it does not use this method at all. In 2007, the document information panel uses InfoPath to render metadata. At the time I naively thought that was a dumb idea because the document information panel cannot display custom column types. For example, let’s say you created a custom column type to hold a phone number with the correct formatting for country code and the like. In a SharePoint site you can display it fine, but in the Office 2007 document information panel, you will never see it. InfoPath only will work with the out of the box in column types.

My rationale was that if Office 2007 instead used JavaScript and HTML, then it should be able to display these sorts of custom columns. That may actually be true, but it is irrelevant. When you think of the sheer number of ways that the rendering of the code could be interfered with (Temporary Internet Files being one precedent), you can see why the Office team might have opted for the safety of InfoPath instead.

Now that I understand that this is how Office 2003 does it, I can plainly see that it leaves too much room for error.

So even though my problem was solved by a simple clearing of the cache, it still remains that things like a bad add-in or a bad AD policy setting could interfere with the Office 2003/SharePoint integration. So if you ever have this issue crop up, try the temporary internet files thing or reset IE’s configuration completely. It may save you a lot of time and pain troubleshooting.

Thanks for reading

Paul Culmsee

www.sevensigma.com.au

<html dir="ltr">

<HEAD>

    <META Name="GENERATOR" Content="Microsoft SharePoint">

    <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">

    <META HTTP-EQUIV="Expires" content="0">

    <!-- _locID="id_PageTitle" _locComment="{StringCategory=TTL}" -->

    <Title ID=onetidTitle>File Properties</Title>

<Link REL="stylesheet" Type="text/css" HREF="/_layouts/1033/styles/core.css">

    <META Name="Microsoft Theme" Content="default">

    <META Name="Microsoft Border" Content="none">

<script src="/_layouts/1033/init.js"></script>

<script src="/_layouts/1033/core.js"></script>

<script src="/_layouts/1033/bform.js"></script>

</HEAD>

<Script ID="Form_Validate">

function Form_Validate(fVisible)

{

    if (frm.FValidate(fVisible))

        document.OWSForm.IsFormValid.value="true";

    else

        document.OWSForm.IsFormValid.value="false";

}

</Script>

<Script ID="Update_UI_From_Values">

var bFormFieldsInited = false;

function Update_UI_From_Values()

{

    frm.SetFirstFocus(bFormFieldsInited);

    bFormFieldsInited = true;

    frm.DataBind();

    DefaultControls();

}

</Script>

<Script ID="Update_Values_From_UI">

function Update_Values_From_UI()

{

    frm.SetFirstFocus(bFormFieldsInited);

    bFormFieldsInited = true;

    frm.FValidate(false);

}

</Script>

<script language="JavaScript">

L_tooltipfile_Text = "";

L_tooltipfolder_Text = "";

selectedElement = null

inChangeSelection = false

slElem = null;

oldSelection = "";

bIsFileDialogView = true;

function selectrow()

{

    if (slElem) {

        slElem.className=oldSelection;

        slElem.title="";

        }

    selectedElement = window.event.srcElement;

    while (selectedElement.tagName!="TR") {

           selectedElement=selectedElement.parentElement;

           }

    slElem = selectedElement;

    oldSelection=slElem.className;        

    slElem.className="ms-selected";

    if (slElem.getAttribute("fileattribute") == "file")

        slElem.title=L_tooltipfile_Text;

    else

        slElem.title=L_tooltipfolder_Text;

    document.selection.empty();

}

function checkScroll()

{

    if (document.body.scrollHeight > document.body.offsetHeight ||

        document.body.scrollWidth > document.body.offsetWidth)

       document.body.scroll="yes";

}

</script>

<BODY marginwidth=0 marginheight=0 onload="checkScroll()" onresize="checkScroll()" scroll=no>

<!-- Banner -->

<TABLE  CELLPADDING=0 CELLSPACING=0 BORDER=0 WIDTH="100%" >

  <TR>

   <TD COLSPAN=3 STYLE="width:100%">

    <TABLE class=ms-bannerframe CELLPADDING=0 CELLSPACING=0 BORDER=0 STYLE="width:100%">

     <TR>

      <TD VALIGN=BOTTOM WITDH=25>

      <font size=1>&nbsp;</font>

      </TD>

     </TR>

    </TABLE>

   </TD>

  </TR>

</table>

<table width=100% cellpadding=4 cellspacing=0>

 <tr>

  <td>

   <TABLE CELLPADDING=0 CELLSPACING=0 BORDER=0 WIDTH="100%" >

    <tr>

     <td>

      <table cellpadding=0 cellspacing=0 style="margin-left: 3px;margin-bottom:2px">

       <tr>

        <td nowrap class="ms-titlearea">Some User</td>

       </tr>

       <tr>

        <td ID=onetidPageTitle nowrap class="ms-pagetitle">My Committee</td>

       </tr>

      </table>

     </td>

    </tr>

    <tr>

     <td>

      <table border=0 width=100% cellpadding=0 cellspacing=0 style="margin-left: 3px;margin-bottom: 3px">

       <tr>

        <td class="ms-sectionline" height=1><img src="/_layouts/images/blank.gif"></td>

       </tr>

      </table>

     </td>

    </tr>

        <!-- Item Form -->

<SCRIPT>

var frm = new OWSForm("OWSForm", false, "http:\u002f\u002fintranetdev\u002fsomesite\u002f_layouts\u002f");

</SCRIPT>

<SCRIPT> frm.dopt.chDateSep = "\u002f"; frm.dopt.chTimeSep = ":"; frm.dopt.SetTimeFormat(0); frm.dopt.SetDateOrder(1); frm.dopt.SetDOW(0); frm.dopt.stAM = "AM"; frm.dopt.stPM = "PM"; frm.dopt.TimeMarkPosn = 0; frm.dopt.webTZOffsetMin = -480;  frm.nopt.chDigSep = ","; frm.nopt.chDecimal = "."; frm.nopt.chMinus = "-"; frm.nopt.iNegNumber = 1; frm.nopt.SetGrouping("3;0"); 

frm.stFieldPrefix = "urn:schemas-microsoft-com:office:office#";

frm.stImagesPath = "\u002f_layouts\u002fimages\u002f";

frm.wBaseType = 1;

</SCRIPT><Form Name="OWSForm" id=OWSForm EncType="multipart/form-data" Action="http://intranetdev/somesite/_vti_bin/owssvr.dll?CS=65001" Method=POST onSubmit="return false;">

<INPUT Type=Hidden Name="_charset_" Value="utf-8">

<INPUT ID=onetidCmd Type=Hidden Name="Cmd" Value="Save">

<INPUT ID=onetidIsFormValid type=hidden name="IsFormValid">

<INPUT ID=onetidFormWasPosted type=hidden name="FormWasPosted">

<INPUT ID="MustUpdateForm" type=hidden name="MustUpdateForm" value="true">

<INPUT type=hidden name="NextID" id="NextID" value="-1">

<INPUT type=hidden name="NextUsing" id="NextUsing" value="">

<SPAN id='part1'><INPUT ID=onetidIOHidden TYPE=HIDDEN NAME="List" VALUE="{EF494645-1D24-4761-A874-0CB866FA494C}">

<INPUT ID=onetidIOHidden TYPE=HIDDEN NAME="ID" VALUE="New">

<TABLE border=0 cellpadding=2>

<SCRIPT>var _g_tp_fNewForm = true;</SCRIPT>

<TR style="display:none"><TH nowrap valign=top class="ms-formlabel"><nobr>Comment<font color=red></font></nobr></TH><TD class="ms-formbody"><SCRIPT>fld = new NoteField(frm,"Comment1","Comment","");fld.stNumLines = "20";fld.IMEMode="";fld.BuildUI();</SCRIPT>&nbsp;<br><SPAN class="ms-formdescription"></SPAN></TD></TR><TR style="display:none"><TH nowrap valign=top class="ms-formlabel"><nobr>Document&nbsp;Status<font color=red></font></nobr></TH><TD class="ms-formbody"><SCRIPT>fld = new ChoiceField(frm,"Corro_x0020_Status","Document Status","Not Started"); fld.format = "Dropdown"; fld.AddChoice("Not Started", "");fld.AddChoice("No Action Required", "");fld.AddChoice("In Progress", "");fld.AddChoice("Completed", "");fld.AddChoice("Deferred", "");fld.AddChoice("Waiting on someone else", "");fld.IMEMode="";fld.BuildUI();</SCRIPT><SPAN class="ms-formdescription"></SPAN></TD></TR><TR style="display:none"><TH nowrap valign=top class="ms-formlabel"><nobr>Content&nbsp;Type<font color=red></font></nobr></TH><TD class="ms-formbody"><SCRIPT>fld = new ChoiceField(frm,"ContentType","Content Type","LSWA Document"); fld.format = "Dropdown"; fld.AddChoice("LSWA Document", "");fld.AddChoice("Folder", "");fld.IMEMode="";fld.BuildUI();</SCRIPT><SPAN class="ms-formdescription"></SPAN></TD></TR><TR style="display:none"><TH nowrap valign=top class="ms-formlabel"><nobr>Committee&nbsp;Document<font color=red></font></nobr></TH><TD class="ms-formbody"><SCRIPT>fld = new ChoiceField(frm,"Committee_x0020_Document","Committee Document",""); fld.format = "Dropdown"; fld.AddChoice("Agendas", "");fld.AddChoice("Minutes", "");fld.AddChoice("Actions", "");fld.AddChoice("Administration", "");fld.IMEMode="";fld.BuildUI();</SCRIPT><SPAN class="ms-formdescription"></SPAN></TD></TR><TR style="display:none"><TH nowrap valign=top class="ms-formlabel"><nobr>Document&nbsp;Type<font color=red></font></nobr></TH><TD class="ms-formbody"><SCRIPT>fld = new ChoiceField(frm,"Document_x0020_Types","Document Type",""); fld.format = "Dropdown"; fld.AddChoice("Incoming Correspondence", "");fld.AddChoice("Outgoing Correspondence", "");fld.AddChoice("Internal Document", "");fld.AddChoice("Fax", "");fld.AddChoice("E-mail", "");fld.AddChoice("Memo", "");fld.IMEMode="";fld.BuildUI();</SCRIPT><SPAN class="ms-formdescription"></SPAN></TD></TR><TR style="display:none"><TH nowrap valign=top class="ms-formlabel"><nobr>Year<font color=red></font></nobr></TH><TD class="ms-formbody"><SCRIPT>fld = new ChoiceField(frm,"Year","Year",""); fld.format = "Dropdown"; fld.AddChoice("2000", "");fld.AddChoice("2001", "");fld.AddChoice("2002", "");fld.AddChoice("2003", "");fld.AddChoice("2004", "");fld.AddChoice("2005", "");fld.AddChoice("2006", "");fld.AddChoice("2007", "");fld.AddChoice("2008", "");fld.AddChoice("2009", "");fld.AddChoice("2010", "");fld.IMEMode="";fld.BuildUI();</SCRIPT><SPAN class="ms-formdescription"></SPAN></TD></TR></TABLE>

<script type="text/javascript">

_g_tp_rgctNames = new Array;

</script><SCRIPT>var _tp_rgctfld = null;var _tp_ctfld = null;var _g_tp_rgcts = new Array;</SCRIPT>

<script type="text/javascript">

_g_tp_rgctNames.push("LSWA Document");

</script>

            <SCRIPT>

            _tp_rgctfld = new Array;

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="ContentType";

                _tp_ctfld.fRequired = BoolFromString("");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="SelectFilename";

                _tp_ctfld.fRequired = BoolFromString("");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="FileLeafRef";

                _tp_ctfld.fRequired = BoolFromString("TRUE");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Created";

                _tp_ctfld.fRequired = BoolFromString("");

                _tp_ctfld.fHidden = BoolFromString("TRUE");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Title";

                _tp_ctfld.fRequired = BoolFromString("FALSE");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("FALSE", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("TRUE", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Modified";

                _tp_ctfld.fRequired = BoolFromString("");

                _tp_ctfld.fHidden = BoolFromString("TRUE");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Modified_x0020_By";

                _tp_ctfld.fRequired = BoolFromString("");

                _tp_ctfld.fHidden = BoolFromString("FALSE");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Created_x0020_By";

                _tp_ctfld.fRequired = BoolFromString("");

                _tp_ctfld.fHidden = BoolFromString("FALSE");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Comment1";

                _tp_ctfld.fRequired = BoolFromString("");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Corro_x0020_Status";

                _tp_ctfld.fRequired = BoolFromString("");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Committee_x0020_Document";

                _tp_ctfld.fRequired = BoolFromString("FALSE");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Document_x0020_Types";

                _tp_ctfld.fRequired = BoolFromString("FALSE");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="Year";

                _tp_ctfld.fRequired = BoolFromString("FALSE");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("");

                _tp_ctfld.stDisplay ="";

                    _tp_rgctfld.push(_tp_ctfld);

                _tp_ctfld = new Object(null);

                _tp_ctfld.stName="MoveDocu";

                _tp_ctfld.fRequired = BoolFromString("FALSE");

                _tp_ctfld.fHidden = BoolFromString("");

                _tp_ctfld.fShowInNewForm = BoolFromString2("", true);

                _tp_ctfld.fShowInEditForm = BoolFromString2("", true);

                _tp_ctfld.fReadOnly = BoolFromString("TRUE");

                _tp_ctfld.stDisplay ="Move Document";

                    _tp_rgctfld.push(_tp_ctfld);

            _g_tp_rgcts.push(_tp_rgctfld);

            </SCRIPT>

<script type="text/javascript">

_g_tp_rgctNames.push("Folder");

</script>

            <SCRIPT>

            _tp_rgctfld = new Array;

            _g_tp_rgcts.push(_tp_rgctfld);

            </SCRIPT>

<script type="text/javascript">

function _FixMpCt2Flds()

{

    var frm = frmCurrent;

    var rgn1 = frm.rgctNames;

    var rgn2 = _g_tp_rgctNames;

    var rgctflds = _g_tp_rgcts;

    if (rgctflds.length < rgn1.length)

    {

        var rgnew = new Array;

        var i;

        var j = 0;

        for (i = 0; i < rgn1.length; i++)

        {

            var n1 = rgn1[i];

            var n2 = rgn2[j];

            if (n1 != n2)

            {

                rgnew.push(new Array);

            }

            else

            {

                rgnew.push(rgctflds[j]);

                j++;

            }

        }

        _g_tp_rgcts = rgnew;

    }

}

_FixMpCt2Flds();

</script>

</SPAN></form>

<SCRIPT>

</SCRIPT>

        <!-- FooterBanner closes the TD, TR, TABLE, BODY, And HTML regions opened above -->

 

</td></tr></table></PlaceHolder></TD></TR>

</TABLE>

</BODY>

</HTML>

No Tags

I’ve been busy on a number of fronts and some of the fruits of that work will appear soon enough, but I thought that I would pop up to let you know about a forthcoming book written by Brett Lonsdale and Nick Swan on a SharePoint component that has until now, been seriously under-represented in the plethora of SharePoint books out there in the marketplace.

The Business Data Catalog is one of those SharePoint components that is easy enough to understand conceptually, but then will scare the utter crap out of you when you delve into the guts of its XML based complexity.  At least that was my experience the first time I toyed with it in early 2007. Luckily for me, my ass was saved by a tool that had just been released as a public beta called BDC MetaMan. I downloaded this tool and within around 15 minutes I used it to set up a BDC connection to Microsoft’s Systems Management Server v4 to pull software package details into a SharePoint list and felt very proud of myself indeed. 

Fast forward to mid 2009 and BDC MetaMan has come a hell of a long way, as have its creators. Nick and Brett are about as world-authoritative as you can possibly get on the BDC and if you wish to become a Jedi in the dark arts of the BDC “force” then you now have your official bible. This book is absolutely crammed with detail and the expertise of the authors in this feature shines throughout.

The book is split up across 11 chapters and although it is not explicitly stated by the authors, seems to be made of 3 broad parts. Chapter 1 introduces the BDC, how it is architected (web parts, BDC column, BDC Search, and integration with User Profile import and the SDK). Also covered is the range of data sources, an introduction to Application Definition Files (ADF) and how it all integrates into the Shared Service Provider model.

Once the intro chapter is done with, Brett and Nick don’t waste too much time in diving deep. 

Chapters 2 and 3 deal with the structure of BDC Application Definition (ADF) files, and follows up with the complex world of how authentication plays out with the BDC. Chapter 2 delves far more into the ADF files than I ever wished to tread, but Nick and Brett somehow manage to describe a long, boring XML file in a logical, easy to follow manner and there was a lot of stuff that I learned here that I had simply missed from trawling MSDN articles. The authentication chapter is covered in excellent detail in Chapter 3 and goes way beyond the usual NTLM/Kerberos double-hop stuff. Authentication in the Microsoft world has become very complex these days, and there are various options and trade-offs. This chapter covers all of this and more, brilliant stuff.

After the deep dive of ADF and authentication, we surface a little from the previous two chapters into what I think really, is part 2 of this book. That is, several chapters that deal with how you leverage the BDC once you have connected to a line of business application. Chapter 4 introduces the built-in web parts that come with the BDC, shows how they are used and how they can be modified either using SharePoint Designer or tweaking XSL styles directly. Chapter 5 explores the BDC column type, how it can be used in the Office document information panel, in SharePoint Designer workflows, as well as its limitations. Chapter 6 explains how to leverage the BDC for allowing SharePoint to crawl your back-end line of business data and present it in search results. In addition to this, chapter 6 has a lot to offer just from the point of view of customising the search experience, whether using BDC or not. Finally, Chapter 7 examines how the BDC can be utilised to add data into user profiles that is leveraged via audience targeting.

Next we dive back into “real programmer” territory and what I think makes part 3 of this book. Chapter 8 delves deep into the BDC object model, for those times when the out of the box stuff just won’t quite cut it for you. The example used to demonstrate this object model is a web service that exposes BDC data via several methods. Chapter 9 then covers the creation of a custom web part that is in effect, an Ajax version of the out of the box “Business Data List web part” that refreshes data every few seconds without requiring a page load. Chapter 10 is particularly interesting because it examines how the BDC is used in conjunction with another oft overlooked suite of technologies known as “Office Business Applications”. The combination of BDC and OBA offer many interesting capabilities and among the examples, there are examples of Excel and Word leveraging the BDC as well as creating custom task panes, custom ribbons and the like. Finally, chapter 11 deals with using the BDC to write data back to the line of business applications and finishes with a great example of using InfoPath to submit data to a line of business application via a webservice that calls the BDC. That is hellishly cool in a nerdy developer kind of a way.

Phew! First up, *man* these guys are smart! I have to say this is the hardest SharePoint book that I have reviewed. It is obviously aimed at developers but it has so much to offer beyond the BDC. The content is very technical at times and obviously low-level. That, itself, is not the problem. Conversely, complex topics are handled really well and everything is extremely logically organised and flows well. The book is simply very, very comprehensive! There is plenty of meat for developers to sink their teeth into and this book will keep you going for a long time.

The preface of the book states that it has been written for an audience of “Microsoft SharePoint 2007 Information Workers and Developers who need to learn how to use, customize and create solutions using the Business Data Catalog”. I would agree with this, but I hope that information workers do not get put off by chapter 2 (and to some extent, chapter 3). This book dives deep straight off the bat and it is actually the middle chapters that offer the sort of insights that information workers will find the most useful.

So, if you think that the BDC deserves more than one single chapter towards the back of a SharePoint book, then this is your answer. As well as becoming an expert on the BDC, It will open your eyes to many possibilities beyond it.

Thanks for reading

Paul Culmsee

No Tags

Welcome to part 4 of my series on demonstrating SharePoint’s usefulness for storing Cisco configuration backups. What a hard slog it’s been! The last article (part 3) of this series focused on how to modify an open source C# TFTP server to upload files into a SharePoint document library using the SharePoint SDK.

Here is the quick recap on what we have covered so far

• Part 1 illustrated how it it possible to use SNMP to tell a Cisco IOS device to dump its configuration to TFTP. We talked about the version control feature of SharePoint and why it makes sense to TFTP your configs to a SharePoint document library. We covered the WEBDAV network provider supplied with XP and Win2003 and finished off with a basic example using the TFTP server TFTPD32.
• Part 2 went into more detail about the issues you can face when using the WEBDAV network provider. It also went into more detail on 3 TFTP server products (WinAgents TFTP, SolarWinds TFTP Server and TFTPD32 and why TFTPD32 ended up being the best choice for this purpose.
• Part 3 then looked at a wonderful open source TFTP server written in C# called TFTPUtil. We modified the source code of this TFTP server to use the SharePoint SDK and upload files to a SharePoint document library.

Now both the WEBDAV and SDK methods had some issues. The WEBDAV method was obviously easy to set up because pretty much any application (theoretically anyway) can be made to work with it. I, however, had reliability issues with this method as part 2 detailed. The SDK method was much more reliable, but had its own problems. Many people would be uncomfortable with having to perform custom modification of an existing open source TFTP server, but more importantly, there are security implications with this method too.

So we have one remaining method that we can explore. This method is still based around modifications to the TFTPUtil source code but instead of using the SharePoint SDK, we instead use the SharePoint web services.

SharePoint 2007 Web Services

SharePoint can be remotely interacted with via Web Services. Out of the box, a number of web services are provided and shown below (along with the URL of the service based around my sample SharePoint server used in this series).

• Administration Web Service (http://tidemo/_vti_adm/Admin.asmx)
• Alerts Web Service (http://tidemo/_vti_bin/Alerts.asmx)
• Authentication Web Service (http://tidemo/_vti_bin/Authentication.asmx)
• Copy Web Service (http://tidemo/_vti_bin/Copy.asmx)
• Document Workspace Web Service (http://tidemo/_vti_bin/Dws.asmx)
• Forms Web Service (http://tidemo/_vti_bin/Forms.asmx)
• Imaging Web Service (http://tidemo/_vti_bin/Imaging.asmx)
• List Data Retrieval Web Service (http://tidemo/_vti_bin/DspSts.asmx)
• Lists Web Service (http://tidemo/_vti_bin/Lists.asmx)
• Meetings Web Service (http://tidemo/_vti_bin/Meetings.asmx)
• People Web Service (http://tidemo/_vti_bin/People.asmx)
• Permissions Web Service (http://tidemo/_vti_bin/Permissions.asmx)
• SharePoint Directory Management Web Service (
• Site Data Web Service (http://tidemo/_vti_bin/SiteData.asmx)
• Sites Web Service (http://tidemo/_vti_bin/Sites.asmx)
• Search Web Service (http://tidemo/_vti_bin/spsearch.asmx)
• Users and Groups Web Service (http://tidemo/_vti_bin/usergroup.asmx)
• Versions Web Service (http://tidemo/_vti_bin/Versions.asmx)
• Views Web Service (http://tidemo/_vti_bin/Views.asmx)
• Web Part Pages Web Service (http://tidemo/_vti_bin/WebPartPages.asmx)
• Webs Web Service (http://tidemo/_vti_bin/Webs.asmx)

So let’s go back to our TFTPutil project in Visual Studio and use Web Services instead of the SDK

Modification Example #2 Web Services

Initially I thought that I was going to have to write my own web service to upload a file to SharePoint, but like TFTPUtil, it didn’t take much googling to find that the hard work had already been done.

In this case, it was the excellent work by txs8311. He had written some articles on how to upload into SharePoint via methods provided by the out of the box web services. We will incorporate his work into TFTPUtil and see how it goes!

Out of respect for txs8311, I will not re-paste his code here, but I will show how I have incorporated it into TFTPUtil.

If you check his blog post, he has created a class called DocLibHelper. We will add this as a class to TFTPUtil and then call it from the TFTPServerProcess class that we modified in my last post.

So let’s go back to our TFTPUtil project in Visual Studio and add a new item to the project. Add a class and call it DocLibHelper.

\

Once added, paste the content of the DocLibHelper class from txs8311′s site. Do not forget to add the references to the libraries used by the code.

using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;
using System.Xml;

Finally, the code interacts with the Lists web service (lists.asmx) and we need to add a reference to a SharePoint web service so we can compile and test the code against the lists web service. From the “Project” menu, choose “Add Web Reference”. In the URL, connect to the lists service for your SharePoint server (in my case http://tidemo/_vti_bin/Lists.asmx)

You MUST name this Web Reference ListsService as txs8311′s code expects it to be this name.

That’s it for the webservice code! Now let’s modify the TFTPServerProcess class to use it! (I go into detail on modifying this class in the previous post).

State.Close();
if (IsSharePoint){
    // Now we add this newly downloaded file to the document library
    try
    {
        DocLibHelper docLibHelper = new DocLibHelper();
        Dictionary<string, object> properties = new Dictionary<string, object>();
        properties.Add("Title", State.Filename);
        string uploadURL = TempPath + @"/" + State.Filename;
        //Create or overwrite text file test.txt in 'Docs' document library creating folder 'Test Folder' as required.
        docLibHelper.Upload(uploadURL, System.Text.Encoding.ASCII.GetBytes(FullPath + @"/" + State.Filename), properties);

        // clean up temp file
        File.Delete(FullPath + @"/" + State.Filename);
    }
    catch (Exception ex)
    {
        AddMsg(Level.Verbose, "Problem opening SharePoint site" + TempPath);
        AddMsg(Level.Debug, "Exception: " + ex.Message + "\n" + ex.StackTrace);
    }
}
// This is the last block so go ahead and setup to delete stateStopListener(); 
StopListener(); 

So let’s have a closer look at this code. Now we are invoking an object based on txs8311′s the DocLibHelper class, rather than the SharePoint SDK classes. After creating an instance of DocLibHelper, we create a dictionary object to hold the properties of the file. We will write the name of the file to the “Title” column (the result of which you will see later).

We then call the Upload method, passing in the URL, a reference to the file to be uploaded and the properties dictionary.

One little gotcha!

One minor thing that I have to mention here, as I got caught by it. Web Services are case-sensitive! The invocation code on the console application was the following:

TFTPServer myTFTP = new TFTPServer();
myTFTP.AllowWRQ = true;
myTFTP.AllowWRQOverwrite = true;
myTFTP.Path = @"http://tidemo/tftp/backups";
myTFTP.StartListener(); 

But in SharePoint the document library is actually called “Backups”. See the problem? I kept getting exception errors telling me “List not found”, and it took me a while to twig to the fact that I had to fix up line 4 of the above code.

myTFTP.Path = @"http://tidemo/tftp/Backups";

Now that oughta do it! Let’s test!

Now we attempt an upload.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

Sweet!

Let’s check the document library. First up view properties so we can see the title field. Very good.

Okay now let’s do a few more TFTP PUT’s.

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

[snip]

C:\>TFTP 192.168.134.129 put c:\myfile.txt
Transfer successful: 43552 bytes in 1 second, 43552 bytes/s

So let’s look at the version history. What do you see now? SharePoint will report changes to properties (column values) in the version history. Version 285 of myfile.txt was the version where I updated the title property.

This method works nicely. The only problem I encountered was a delay after the first running of the program. To connect to the lists webservice, authenticate and call the necessary methods a little while on the first attempt. But subsequent attempts were fine. Since I did this on a VM on a heavily loaded laptop, it may not be noticeable in a prod environment at all.

Modification Example #3, FrontPage Extensions and RPC

txs8311 also wrote a second part to his posts on uploading files into SharePoint. In this post he rewrote his DocLibHelper class to use FrontPage extensions and RPC. I did not try this, as I was satisfied with the modification Example #2. But all you should need to do, is replace the code in the DocLibHelper class with the code from his page.

If you are more keen than me, feel free to give it a shot and let me know how you go!

Summary of Upload Methods

So let’s recap on the three TFTP upload methods that we have used and summarise the advantages and disadvantages of each.

CleverWorkArounds Rating: Rock on SharePoint Web Services!

Additional Coolness

Since we have modified this TFTP Server to our requirements, you are pretty much free to add SharePoint specific features to make integration even nicer. One feature that springs to mind is setting more metadata properties when the files are uploaded. Here is an example. Let’s assume that we have a list of Cisco IP addresses and the MODEL and TYPE of device on that address. E.g 6509 Switch, 3725 Router or Aironet 350 Wireless Access Point.

It would be pretty easy to modify our TFTP server to be able to examine this list when receiving a file and based on the IP of the sender of the file, write the type of device to a “Device Type” column in our configuration backups library.

Better still, it would then be very cool if that list of IP’s and device types already exist in SharePoint. Imagine this list exists in the form of an IP address register or better still, an IT device asset register (along with serial numbers, maintenance contract information and the like). It would be easy for staff to maintain this information and the list would have version control just like our “Backups” document library. But for me what would be really nice about this method is the organisational efficiency that would be gained.

That gain is this: The person that updates the asset register list isn’t necessarily the Cisco engineer that sets up the device. Critical device information is entered ONCE and leveraged from then on. The next post in this series will expand on these benefits.

Conclusion

Right! We are DONE with getting files into SharePoint and about damn time too!

The next posts in this series will move away from application development and infrastructure issues and move into some higher level governance issues. The aim here is to show Cisco and IT infrastructure people what you can do with SharePoint once you get the data into SharePoint. I hope if I do a decent job of writing it, that it will be easy to see just how this can be re-used for any IT infrastructure management.

But for now, I bid you adieu until then….

No Tags