Ever noticed a tweet disappear? As in someone else can't see something you posted? There's a common mistake even twitter pros make that causes this to occur.

Are you sure you're seeing all of the replies to your tweets -- even from people you don't follow? Do you ever find yourself missing important posts -- like those from your real life friends? Do you know which will reach a wider audience: new style retweets vs old style retweets?

This article will answer these and other twitter related questions. It will help beginner to intermediate twitter users get the most out of the service.

#1 Standard #Tweets

These appear simple. These show up in the timeline of your followers (@admirer and @vet in the example above). And they don't show up in the timeline of anyone who doesn't follow you (@effusive). If this was all there was to twitter it would be a complete waste of time.

The subtlety with this type of tweet is that your content may show up to someone that doesn't follow you but that is following a hashtag or a topic you mention. In the example above @hiro thows a hashtag onto the word aardvarks to indicate that it's special somehow. His tweet is then picked up by @spca who runs a constant search in a twitter client for the word "#aardvark".

The first time I realized how powerful this could be I had was at a conference. I was extremely bored by a terrible keynote. With nothing else to do I discovered the conference hashtag on twitter and started watching it. Suddenly I realized I wasn't alone. I was in a huge room full of people making fun of the presenter! It was like telepathy, or group consciousness or something. I've never felt alone or bored at a conference since.

Interesting statistic: As of June 2010, about 65 million tweets were posted each day, equaling about 750 tweets sent each second. (reference)

#2 Replies

Replies are more complicated. In the example above the guy on the left, @vet, replies to @hiro by prefixing his tweet with "@hiro". However, this probably doesn't work like you think. The tweet obviously shows up in @hiro's timeline, but it also shows up in @admirer's timeline because she follows both people mentioned in the tweet (@hiro and @vet). But the tweet does not show up to @effusive because, while he follows @vet, the author of the tweet, he doesn't follow @hiro, the person being mentioned.

Getting this scoping rule wrong is way too easy. For instance you can't just announce that someone did something great. If @hiro started a tweet with "@vet gave a great anti-aardvark presentation" then most of his followers (e.g. @effusive) wouldn't actually see the tweet! I've seen seasoned twitter users make this mistake.

As an interesting aside twitter can help track conversations. If you click a reply button in a twitter client or on the website twitter will keep track of which tweet you replied to and then help piece a conversation together. But if you simply start a new tweet with "@somone" without clicking reply, the above scoping rules still apply, but twitter won't help anyone reconstruct the conversation.

Finally a word on notifications: If someone mentions you (via a reply or anywhere in their tweet) and you follow them then you will get an e-mail and possibly an SMS notification. If someone that you do not follow replies to you, you will not get an email. For this reason you must either frequently check the "mentions and more" section of the website or use a column based twitter client, discussed in the final section on lists.

Interesting statistic: On average only 23% of tweets get a reply. (more)

Another one: 38% of tweets are conversational. In other words 38% of tweets start with an "@". If your percentage of replies is lower you probably aren't using twitter to its fullest. (more)

#3 Reply-All's

So what happens if you want to reply to someone, but still have it show up in the timelines of all of your followers? As long as your tweet doesn't start with an "@" symbol then you're fine. So the convention that has grown up is to prefix your tweet with a "." and then the "@someone". In the example above @effusive now sees the tweet where he didn't in a standard reply.

Interesting statistic: The highest usage rate of twitter to date occurred during the 2011 FIFA Women's World Cup Final between Japan and the United States when 7,196 tweets were published per second! (more)

#4 Old Style Retweet or "Retweet with Comment"

Prior to late 2009 if you saw a tweet you wanted to share with your followers you would retweet (RT) it by copy and pasting their tweet and prefixing it with "RT @author". The problem from a scoping perspective was that people would get duplicates. In the example above @admirer sees a duplicate from @vet when he retweets @hiro's tweet.

There are other problems with old style retweets unrelated to scoping. One problem is that you get long chains of "RT @person1 RT @person2 ...". Another problem is it was hard to follow a person but not all their retweets. The biggest problem is old style retweets take valuable characters away from your available 140, and sometimes require mangling the original.

Old style retweeting is still in use today primarily from people wish to retweet but add a comment, but also because it allows people to reach a wider audience (since new style retweets won't show up in searches or lists, more on this later).

Interesting statistic: Only 6% of tweets get retweeted (more)

#5 New Style Retweet

In November of 2009 twitter rolled out a new technique for retweeting aimed at solving the problems mentioned above. It caused a lot of confusion, but did solve the problems.

The end result of new style retweets from a scoping perspective is that sometimes people see tweets from people they don't follow. In the example above @effusive sees @hiro's tweet in his timeline even though he doesn't follow @hiro, because @vet performed a new-style retweet.

If you click the retweet button on the twitter website today it will perform a new style retweet. Some twitter clients like TweetDeck give you the option. The only way to perform an old style retweet or retweet with comment via the website is to copy and paste the message.

Interesting statistic: Less than half of tweets are posted using the web user interface. By far the most common twitter client is Tweetdeck with 8.48% of the market. (more)

#6 Direct Messages

Direct messages are simple. Prefix your tweet with "D somone" (no at symbol before the name) and they will be the only person that will see it. Just like e-mail except the recipient will get an e-mail and a text message if they've set that up on their phone. Note that neither people watching an included hashtag (e.g. @spca) nor people mentioned will see direct messages unless they are the recipient.

Interesting statistic: Six percent of all tweets are sent via SMS. (more)

#7 Lists

The last scoping topic worthy of mention is lists. This fantastic feature was added by twitter in late 2009. It allows you to organize the people you follow. For instance I have a private "Infrequent Posters" list that I try very hard to never miss a tweet from. Meanwhile I follow enough people that my main list is more like a stream that I dip into but don't get stressed out if I miss some of.

Lists work best when used in conjunction with a column based twitter clients (e.g. TweetDeck). These types of twitter clients are the only way to go if you want to become a twitter pro. If you've gotten to the point where you start missing content from important people (e.g. your real life friends), or you're missing responses to your tweets, or mentions from people you don't follow: then you probably need lists, but you absolutely need something like TweetDeck.

In regards to scoping, lists contain one subtlety that you need to be aware of. They do not support new style retweets. In the example above @vet posts a normal tweet. He then new-style retweets one of @hiro's tweets. @effusive follows @vet and has also placed him into a list. While the list helps @effusive not miss tweets from @vet, it does not end up displaying @hiro's tweet.

Note that this rule applies both to people who maintain their own lists as well as to people who follow lists maintained by other people (e.g. your company may maintain a public list of its twitter enabled employees).

Interesting Statistic: Pointless babble is the most common type of content on twitter. It represents 40.55% of all traffic. (more)

Conclusion

Twitter may appear to be fairly simple at first glance. In reality it has grown over the years to become very sophisticated. Sophisticated and incredibly powerful. But to fully tap that power you must understand the scoping rules and subtleties surrounding things like replies, retweets, and lists. Hopefully you're closer now.

Have you ever had an enum in your server code that you wanted to access in client side code? Or wanted the safety of compile time errors when there are discrepancies between your server-side and client-side code? Or had a C# Data Transfer Object (DTO) that you wanted to enable Intellisense for in JavaScript? I hadn't found a good solution either ... until now.

Compiling C# to What!?

Compiling C# code into JavaScript may seem foreign, but Script# is a mature technology that is absolutely worth a look. Our team has been using it very happily for about three months. We've found a number of benefits including:

• Consistently working Intellisense
• Better encapsulation
• Simpler Object Orientation
• Easier deployment (Script# compiles multiple files to a single, optionally minified file)
• Safer refactoring
• Simpler unit testing
• Type safety; and
• Design time syntax checking (no more mistyping a variable and accidentally declaring it to the global scope)

Scott Hanselman covered most of these topics in last month's Hanselminutes episode: Script# Compiles to JavaScript. If you have a spare car ride I definitely recommend the listen. But what wasn't covered were some additional benefits provided by an off the beaten path approach to this great technology. The main benefit is tighter server side C# to client side JavaScript integration. A secondary benefit is less of a dependency on Script#.

To the Command Line

Typically to get going with Script# you:

1. Install the Script# Visual Studio plug-in
2. File -> New project
3. Select "Script Library" and
4. Compile to generate JavaScript

Nice and easy. The down side to this approach is you can't easily include files outside of your project. Specifically, you can't include data transfer objects or enum's from your server side code. Furthermore, all of your team members must install Script#. If instead you compile with Script#'s "ssc.exe" command you obtain more control and get less dependency.

How-To

The how-to looks something like this:

1. Download and install Script#
2. Add a Class Library (not a Script# project)
3. Project properties -> Build -> Advanced -> "Do not reference mscorlib"
   
4. Ideally move the Script# files (C:\Program Files (x86)\ScriptSharp) locally to the solution and check them into source control to a Libs or something similar
5. Remove all references, but add: ScriptSharp.dll, ScriptSharp.Web.dll, Script.Jquery
6. Edit your .csproj to manually reference Script#'s mscorlib (right click, Unload project, Edit MyProject.csproj)
   
   <Reference Include="mscorlib, Version=0.7.0.0, Culture=neutral, PublicKeyToken=8fc0e3af5abcb6c4, processorArchitecture=MSIL">
     <SpecificVersion>True</SpecificVersion>
     <HintPath>..\Libs\ScriptSharp\v1.0\mscorlib.dll</HintPath>
   </Reference>
   
   
7. Modify AssemblyInfo.cs and remove the following lines:
   
   [assembly: ComVisible(false)]
   [assembly: Guid("b5e2449f-193c-46d1-9023-9143618d8491")]
   
   
8. Modify AssemblyInfo.cs and add the following:
   
   [assembly: ScriptAssembly("ScriptSharpDemoAssembly")
   
   
9. Ensure it compiles in Visual Studio
10. Create a batch script or PowerShell script that compiles using ssc.exe like this:
    
    ..\Libs\ScriptSharp\v1.0\ssc.exe ^
        /debug ^
        /out:MyScriptSharp.js ^
        /ref:"..\Libs\ScriptSharp\v1.0\Framework\mscorlib.dll" ^
        .\Properties\AssemblyInfo.cs ^
        .\Class1.cs
    
    

Notice the last two lines in the script are a list of the files that you want to include. The point of this exercise is that you can now include files outside of your main Script# project in that list.

Now for completeness if you put a simple static method in Class1.cs, something like this:

namespace MyScriptSharp
{
    public class Class1
    {
        public static string HelloWorld()
        {
            return "Hello World!";
        }
    }
}

Then run the batch file you should get something like this:

Type.registerNamespace('MyScriptSharp');

////////////////////////////////////////////////////////////////////////////////
// MyScriptSharp.Class1

MyScriptSharp.Class1 = function MyScriptSharp_Class1() {
}
MyScriptSharp.Class1.helloWorld = function MyScriptSharp_Class1$helloWorld() {
    return 'Hello World!';
}

MyScriptSharp.Class1.registerClass('MyScriptSharp.Class1');

Obviously you could get these results faster with approach #1. But now you have a lot more control.

Summary

The main downside to this approach is maintaining the batch file is a bit of a hassle. But the upsides are that you can include any file from your server-side C# code. And any changes in that server-side code are automatically reflected in your JavaScript. And any breaking changes in your server-side code generate compile time errors in your client side code. And furthermore none of your team members need to install Script#. For our team it's an easy tradeoff. What about for yours? Please share your thoughts in the comments or on twitter.

If you've read my previous posts on the subject you know that I love agile burn up charts for managing SCRUM style projects, particularly compared to burn down charts.

The problem is that burndown charts lack two essential pieces of information. First, how much work was actually accomplished during a given iteration (as opposed to how much work remains to be completed) and second how much total work the project contains (or if you prefer how much scope has increased each iteration).

What you might not know is how flexible they can be. In particular the example I gave last time has a problem. Can you spot it?

The problem is that the scope increase of 20 points in iteration six might very well have had zero impact on the actual deployment date. Those points might have all been low priority tasks that the team will work on after they deploy the initial 100 points. Wouldn't it be nice if the burn up chart could convey priority information as well?

So that's exactly what I did on my current project last week, and it appeared to be a big hit. And with the addition of some Excel trend lines the chart was able to convey a lot of insight into the project:

Obviously all that data is made up, but the chart still tells an interesting story:

• The customer hasn't been adding high priority tasks, perhaps because there is a hard deadline approaching
• Because of the customer's restraint in adding high priority tasks the team can be fairly confident that they will finish all the high priority tasks by Iteration 3
• While the customer hasn't added high priority tasks they have still been able to be agile by adding normal and low priority tasks

What's really wonderful about this way of reporting is that it gives the customer the flexibility to add scope to future iterations while maintaining near term deadlines. In short it supports what agile is supposed to be about.

But why stop there? Depending on the story you need to tell or the scenario you need to evaluate perhaps you could incorporate team member contributions. For example:

In this example we can try to show what might happen if we remove a particular team member or if we'd never had a particular team member. What's unique about this way or reporting is that it's incorporating the backlog's increasing trend into account. While it's possible that the backlog may increase or decrease at a different rate with a different team makeup, it's a better scenario than ignoring the rate of increase of the backlog all together.

Perhaps we could incorporate other information. Maybe bug vs. feature information. Or planned vs. actual (e.g. task slippage) information. It seems like there is a lot of potential. Any other ideas out there? Feel free to post in the comments or via twitter. If there's enough interest I'd be happy to post another screencast similar to the last one I did on how to produce burn up and burn down with SharePoint and Excel except this time with trends and priorities.

So overall while these charts may be a little more complicated than a traditional burn down chart they would make an excellent talking point during a PowerPoint presentation, or as part of your iteration close out. And with a little training I bet just about any customer would learn to love the insight they provide.

Occasionally I hear complaints that LINQ is hard to unit test. These complaints aren't about LINQ to objects, mind you, they're specific to the complexities of the flavors of LINQ that turn C# code into something else like SQL or CAML using expression trees. The most common technologies are LINQ to SQL, the Entity Framework, or in my case at the moment LINQ to SharePoint. In this post I'm going to propose a technique that makes testing LINQ not just easy, but downright elegant - assuming you're ok with extension methods - lots of extension methods. And assuming you're ready to kill your Data Access Objects (DAO) tier.

The Unit Testing Problem

Any architecture needs a place to put code that finds entities. For instance FindBySocialSecurityNumber(). In a traditional architecture we might put a method like this is in a DAO layer. If so our method will look something like this:

public class EmployeesDao {
    public Employee FindBySSN(Context ctx, string ssn) {
        return ctx.Employees.SingleOrDefault(e => e.Ssn == ssn);
    }
}

So how would we go about unit testing this?

One fairly typical solution would be to use an in-memory database. That approach works if our data store is a database, but it certainly doesn't work if the data store is something less traditional like SharePoint. But even if our store is a database, we'll still have the hassle of setting up the in-memory database.

Another solution might be to use a mock Context that returns an IQueryable. But wouldn't it be wonderful if we could avoid mocking all together?

Killing the DAO

The first question is why we even have a DAO tier to begin with. The original idea was that we wanted a place to put code specific to a particular data store. In other words we wanted to isolate the code that will need to be changed should the data store switch from SQL Server to Oracle. But isn't that exactly what LINQ does? I'd be pretty surprised if there wasn't a decent LINQ provider for just about any data store at this point that required more than minimal code changes. So why not embrace LINQ and reconsider alternatives to a DAO tier?

One alternative that I've been using for over a month now is to switch to extension methods. To give credit where it's due the idea originated with a conversation with fellow Near Infinity employee Joe Ferner. And I'm sure the idea isn't particularly original (please post in the comments if you know others that use this approach).

Using this technique our code changes from something like this:

var employeeDao = new EmployeesDao(); // or use IOC of course
employeeDao.FindBySSN(ctx, "111-11-1111");

To something like this:

ctx.Employees.FindBySSN("111-11-1111");

Among other things I find this far more aesthetically pleasing because each of the three elements to the statement represent a subsequent filtering of data. It's a more functional way of looking at things.

We could implement this off of the Employees property of the context if we have control over that (which I don't with spmetal). But if we implement this as an extension method like this:

public static class EmployeeExtensions {
    public static Employee FindBySSN(this IQueryable<Employee> employees, string ssn) {
        return employees.SingleOrDefault(e => e.Ssn == ssn);
    }
}

We now have something that's considerable easier to unit test.

Testing It

Once we've refactored our function as an extension method that filters down the corpus of entities, we can test the code using in-memory objects with a call to .AsQueryable(). For instance:

public void FindBySSN_OneSsnExists_EmployeeReturned() {
       var employees = new [] { new Employee { Ssn = "111-11-1111" } };
       var actual = employees.AsQueryable().FindBySSN("111-11-1111");
       Assert.IsNotNull(actual);
}

Notice we didn't have to mock anything.

Testability, but at What Cost?

This technique works great for the example above, but how does it scale to harder problems and what other downsides are there?

As far as scalability I've found this technique works great for every scenario I've run across in the month I've been doing it. It works for joins, aggregations, and even for inserts, update, and deletes.

As far as downsides the astute reader may be wondering about mockability. For instance what if we want to mock the call to FindBySSN and give it the exact Employee that will be returned. This scenario is admittedly harder. But what I've found is that far more often than not I don't really need to mock the types of things that used to live in the DAO tier. Instead I just mock the Employee object off of context to return in-memory objects and make my tests slightly larger in scope. Most of the time I find the larger scope increases the usefulness of the test. In the occasional case where I do really want to mock the "DAO" tier I use a technique described in this post by Daniel Cazzulino.

Conclusion

Obviously there is more to this architecture, for instance how do you handle insert and update operations? The short answer is it's easy, but I'll save that topic for a future post. For now why not give this approach a try? You weren't really happy with that useless old DAO tier anyway, were you? I say we eradicate it and never look back.

This is part 4 of a multi-part series exploring building client side AJAX applications with ASP.Net AJAX 4 Templating and the WCF Data Services (aka ADO.Net Data Services, aka oData, aka Astoria) in SharePoint 2010.

Part 1 - Exploring WCF Data Services in SharePoint 2010
Part 2 - Creating a read-only page with ASP.Net AJAX 4.0 Templates
Part 3 - Writing data back to SharePoint with ASP.Net AJAX 4.0 Templates
Part 4 - jQuery and Manual JSON Object Manipulation

In the previous posts in this series I've shown how you can access SharePoint data using WCF Data Services, how you can display that data using ASP.Net AJAX 4.0 Templates, and how you can write data back to SharePoint.

In this post I'll take writing data back to SharePoint a step further by showing how you can modify non-visible fields by modifying JSON objects directly. More specifically I'll enable dragging user story index cards on a virtual bulletin board using a jQuery plugin, save the X and Y coordinates back to JSON objects, and then save the updated records back to SharePoint.

Cards on a Corkboard

The first thing I'll do is style the cards and display them with absolute position. While I'm at it I'll also reference the draggable jQuery UI plugin. If you're following along at home just built a custom .js download from the jQuery UI site, drop it in your layouts directory and add a reference. So my PageHead content control now looks like this:

<asp:content
    id="PageHead"
    contentplaceholderid="PlaceHolderAdditionalPageHead"
    runat="server">

    <script type="text/javascript"
        src="../ajax/MicrosoftAjax.js"></script>
    <script type="text/javascript"
        src="../ajax/MicrosoftAjaxDataContext.js"></script>
    <script type="text/javascript"
        src="../ajax/MicrosoftAjaxTemplates.js"></script>
    <script type="text/javascript"
        src="../ajax/MicrosoftAjaxAdoNet.js"></script>
    <script type="text/javascript"
        src="../jquery-ui-1.8.custom/js/jquery-1.4.2.min.js"></script>
    <script type="text/javascript"
        src="../jquery-ui-1.8.custom/js/jquery-ui-1.8.custom.min.js">
    </script>
    <style type="text/css">
        .sys-template
        {
            display: none;
        }
        .userstorycard
        {
            border: 1px solid #777777;
            width: 200px;
            position: absolute;
            cursor: move;
        }
        .carddescription
        {
            font-size: 13px;
            background-image: url('card_bg.jpg');
            padding: 0px 5px 5px 5px;
        }
        .cardtitle
        {
            font-size: 15px;
            font-weight: bold;
            padding: 0px 0px 0px 0px;
            border-bottom: 1px solid red;
            background-color: White;
            padding: 0px 5px 0px 5px;
        }

The next thing to do is add X and Y fields to the user story list in SharePoint. And finally display the cards with absolute positioning:

Notice the sys:style attribute. The sys: part is the namespace you need to add for attributes when using templating that I mentioned in part three. Also notice the JavaScript string concatenation inside the binding. You can put in any JavaScript in there that you like. So now if you manually set X and Y values in SharePoint for a card the card shows in the right place. But we're still missing the ability to drag cards.

Making Cards Draggable with jQuery

In order to enable dragging support you're supposed to call the draggable() function on the DOM elements you want to make draggable. Simple right? You just call $(".userstorycard").draggable() which should find every element with a class of userstorycard and call draggable() on it.

But if you do this onLoad() then ASP.Net AJAX Templating engine hasn't had a chance to render the new DOM elements yet. Fortunately the DataView has the JavaScript equivalent of an OnRendered event that you can tie into:

function onLoad() {
    dataContext = $create(
        Sys.Data.AdoNetDataContext,
        { serviceUri: "/demoprep/_vti_bin/ListData.svc" }
        );

    dataView = $create(
        Sys.UI.DataView,
        {
            autoFetch: true,
            dataProvider: dataContext,
            fetchOperation: "UserStories",
            fetchParameters: { $top: 20 }
        },
        {
            rendered: onRendered
        },
        null,
        $get('userStories')
    );
}

function onRendered() {
    // from http://jqueryui.com/demos/draggable/
    $(".userstorycard").draggable();
}

And that's it for enabling dragging. Pretty simple. The result looks like this:

Ok, well you can't see the dragging support, but it's there, trust me. And it's pretty cool except for one thing.

Saving Non-Visible Properties

The problem is that every time you refresh the page the nice layout you've done disappears and everything goes back to being stacked one on top of the other in the upper left. My first thought was to create input type=hidden form elements on the page. This approach causes a lot more plumbing code than is necessary. The better way is that you can access the underlying in-memory JSON objects. Step one is to register an "event" for when the user stops dragging a card.

function onRendered() {
    // from http://jqueryui.com/demos/draggable/
    $(".userstorycard").draggable({
        stop: onDragStop
    });
}

Once you wire that up the onDragStop() function is where all the interesting stuff happens:

function onDragStop(event, ui) {
    var userStoryCard = ui.helper[0];
    var selectedUserStoryJsonObject =
        dataView.findContext(userStoryCard).dataItem;

    var newX = ui.position.left;
    var newY = ui.position.top;

    Sys.Observer.setValue(selectedUserStoryJsonObject, "X", newX);
    Sys.Observer.setValue(selectedUserStoryJsonObject, "Y", newY);
    dataContext.saveChanges();
}

The first line gets the DOM element that just the user just completed dragging. The second line retrieves the in memory JSON object represented by that DOM element. The next couple of lines get the X and Y that the user story card was dragged to relative to their parent DOM element. And then something odd happens.

Sys.Observer.setValue looks so complicated. Why couldn't we just call selectedUserStoryJsonObject.X = newX?

Well, if we'd done that the DataView wouldn't know about the change we just made. If we then tried to call dataContext.saveChanges() nothing would get sent back to the server. Sys.Observer.setValue notifies any interested parties, in this case the DataView, that a value has been changed. And this enables the functionality we saw in part two where the DataView only sends the relevant records back to SharePoint instead of the entire set of JSON objects.

And so, with this code in place, we can now move user story cards around on the page and when we refresh the page or come back days later our layout has persisted back into SharePoint.

Conclusion

In this post you've seen how to directly modify the in-memory JSON objects and send them back to SharePoint. Our application is finally getting interesting and I find it quite remarkable how little code it's taken. In the next post I'll show how to do master-detail scenario's for editing user story cards.