MICHEL'S WEBLOG

Windows Azure Table Storage in the development environment, part II

2010-08-20T14:33:32.4580446+02:00

In addition to my previous post Windows Azure Table Storage in the development environment I’ve created a demo application in which responsibilities are distributed over classes in a way that hides the implementation details of Azure’s table storage.

The application is pretty simple, a one page ASP.NET app in which you can add/edit movies. The movies are displayed on the same page in a GridView.

Movie model
The model has changed a bit. The RowKey is now filled with a GUID (instead of the title) to uniquely identify a movie in the table. The title now has a dedicated property. The PartitionKey remains the movie category as this helps Azure to partition the data over multiple servers if the load gets too heavy.

StorageContext

The StorageContext is the actual entry point to the data. It inherits from TableServiceContext and provides a property Movies that returns an IQueryable of type Movie, which can be used to query the data from storage. It also provides an AddMovie method to avoid exposure of the table name that is required for using TableServiceContext.AddObject.

To create a new StorageContext the StorageContextFactory can be used, which hides the creation/initialization of the StorageContext.

MovieRepository
The repository is used to access the data from the application. Creation of a MovieRepository requires a context that implements IStorageContext (e.g. StorageContext), which can be obtained by the StorageContextFactory. The reason for using an interface to the context is to be able to replace the context by some other implementation, for example in case of a non-Azure application or in case of unit-testing.

Example

             1: MovieRepository repository
=

             2: 
            new MovieRepository(StorageContextFactory.GetContext());

3:

             4: 
            // Get all movies

             5: IEnumerable allMovies = repository.GetAll();

6:

             7: 
            // Get a single movie

             8: Movie movie = repository.GetById(key);

9:

             10: 
            // Add a new movie

             11: Movie newMovie = MovieFactory.Create();

             12: newMovie.PartitionKey = "Action";

             13: newMovie.Title = "An action
movie!";

             14: repository.Add(newMovie);

A few notes:

A MovieFactory is used to create a new Movie. In this factory an instance of a Movie is created and the RowKey property is populated with a new GUID. There are several alternatives but in this way you avoid unnecessary creation of GUIDs when objects are created by the context (in case you initialize in the Movie’s constructor), and you avoid the need for an initialization method that can potentially be forgotten to invoke.
The Add and Update methods on repository directly save to the storage. This might not be what you want in other apps, but for this example (with only one update in each postback) it is good enough.

The source of the example can be downloaded here.

Windows Azure Table Storage in the development environment

2010-08-20T01:36:47.6434761+02:00

Lately I’ve been playing around with Microsoft Azure (for now in the offline development environment only) and it is really easy to get started. However, as easy it is to get your ASP.NET app running, it is a bit more difficult to get access to the storage part (blobs, table storage and queues). There is no straight forward how-to available, at least I couldn’t find one. So I’ve combined some of the resources I’ve found (e.g. the example from Windows Azure Table Storage – Not Your Father’s Database, by Julie Lerman and the PDC09 presentation Windows Azure Tables and Queues Deep Dive, by Jai Haridas) and here it is…

Step 1
Create a new ‘Windows Azure Cloud Service’ project and name it something like CloudServiceDemo, hit ok. Then select an ‘ASP.NET Web Role’ to be included in your cloud service solution and again hit Ok. Now two projects are created, one with roles and configuration and one with the actual ASP.NET web site.

When you run this the ‘Windows Azure Simulation Environment’ starts and your web application appears in your browser (note: at this moment my Queue storage fails with an error that it is being used by another process, something which I haven’t figured out yet).

Step 2
Change two configuration files:

In the ServiceDefinition.csdef, add line 3 in the configuration settings:
```
 1: 
```
```
 2:  "DiagnosticsConnectionString" />
```
```
 3:  "DataConnectionString" />
```
```
 4: 
```
In the ServiceConfiguration.cscfg, add line 3 in the configuration settings:
```
 1: 
```
```
 2:  "DiagnosticsConnectionString"value="UseDevelopmentStorage=true" />
```
```
 3:  "DataConnectionString"value="UseDevelopmentStorage=true" />
```
```
 4: 
```

Step 3
Add the following code to the OnStart method of the WebRole.cs

             1: 
            // Setup
a handler to update CloudStorageAccount instances when the

             2: 
            // configuration settings change
in the service configuration file.

             3: CloudStorageAccount.SetConfigurationSettingPublisher(

             4:  (configName, configSetter) =>

             5:  {

             6: 
            // Provide the configSetter
with the initial value

             7:  configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));

8:

             9:  RoleEnvironment.Changed +=

             10:  (sender, arg) =>

             11:  {

             12: 
            if (arg.Changes.OfType()

             13:  .Any((change) => (change.ConfigurationSettingName
== configName)))

             14:  {

             15: 
            // The corresponding configuration
setting has changed, propagate the value

             16: 
            if (!configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)))

             17:  {

             18: 
            // In this case, the change
to the storage account credentials in the

             19: 
            // service configuration is
significant enough that the role needs to be

             20: 
            // recycled in order to use
the latest settings. (for example, the

             21: 
            // endpoint has changed)

             22:  RoleEnvironment.RequestRecycle();

             23:  }

             24:  }

             25:  };

             26:  }

             27: );

Step 4
Create a class that represents the data in the table storage (you need to add a reference to System.Data.Services.Client)

             1: 
            ///

             2: 
            /// Representation of a movie

             3: 
            ///

             4: 
            /// PartitionKey,
RowKey and TimeStamp are required for Azure table storage.

             5: [DataServiceKey("PartitionKey", "RowKey")]

             6: 
            public
            class Movie

             7: {

             8: 
            ///

             9: 
            /// Category of the movie, e.g.
Action, SciFi, etc.

             10: 
            ///

             11: 
            public
            string PartitionKey
{ get; set; }

12:

             13: 
            ///

             14: 
            /// Title of the movie

             15: 
            ///

             16: 
            public
            string RowKey
{ get; set; }

17:

             18: 
            public DateTime TimeStamp
{ get; set; }

19:

             20: 
            public
            int ReleaseYear
{ get; set; }

             21: }

Step 5
Now you can create the ‘Movie’ table using a CloudStorageAccount

             1: CloudStorageAccount storageAccount
=

             2:  CloudStorageAccount.FromConfigurationSetting("DataConnectionString");

3:

             4: storageAccount.CreateCloudTableClient().CreateTableIfNotExist("Movies");

Step 6
And access the storage using a TableServiceContext

             1: 
            // Create
a context

             2: TableServiceContext context =

             3: 
            new TableServiceContext(storageAccount.TableEndpoint.ToString(),
storageAccount.Credentials);

4:

             5: 
            // Add movie

             6: Movie newMovie =

             7: 
            new Movie { PartitionKey
= "Action", RowKey = "An action movie",
ReleaseYear = 2005 };

             8: context.AddObject("Movies",
newMovie);

             9: context.SaveChanges();

10:

             11: 
            // Get movies

             12: IEnumerable movies = from m in context.CreateQuery("Movies")

             13: 
            where m.PartitionKey
== "Action"

             14:  select m;

Of course this is just a start, now the code has to be placed into specific classes dividing the responsibilities. Something like a StorageContext (which inherits from TableServiceContext) to encapsulate the tablename and the context.CreateQuery, and a MovieRepository to encapsulate the context and the get-, add- and update-methods…

Update: see Windows Azure Table Storage in the development environment, part II for an example.

Fluent test data builder for unit tests

2010-07-26T15:29:00+02:00

A while ago I came across the presentation Sustainable Test-Driven Development by Steve Freeman. In this great presentation Steve talks about writing unit-tests and how to avoid the danger of them growing to huge, complex, hard to read and hard to maintain pieces of code. One of the solutions he talked about was using ‘test data builders’ to compose an object graph of test data in a fluent way, which greatly improves the readability of the unit-test. For example:

             1: Order o = anOrder()

             2:  .WithOrderDate(DateTime.Now)

             3:  .WithCustomer(aCustomer()

             4:  .WithName("John Doe")

             5:  .WithCity("Somewhere")

             6:  .Build())

             7:  .WithOrderLine(anOrderLine()

             8:  .WithProduct(aProduct()

             9:  .WithDescription("Windows Phone
7")

             10:  .WithPrice(499)

             11:  .Build())

             12:  .WithQuantity(2)

             13:  .Build())

             14:  .Build();

My first thought was: great, but do we really need this in C# (the presentation was Java) since we have the new type initialization introduced in C# 3.5? Creating the same object graph using C# is very similar, e.g.:

             1: Order o = new Order()

             2:  {

             3:  OrderDate = DateTime.Now,

             4:  Customer = new Customer()

             5:  {

             6:  Name = "John Doe",

             7:  City = "Somewhere"

             8:  },

             9:  OrderLines = new List()

             10:  {

             11: 
            new OrderLine()

             12:  {

             13:  Product = new Product()

             14:  {

             15:  Description = "Windows Phone
7",

             16:  Price = 499

             17:  },

             18:  Quantity = 2

             19:  }

             20:  }

             21:  };

However, this means that you have to make the set-method of the OrderLine collection public. If you don’t want that (or can’t because you use something like EF, in which collections are created internally) you end up with creating a new order and adding each new order line afterwards.

Which syntax is better? That is really a matter of personal opinion, but the builder solution comes with additional benefits like:

default values of properties (e.g. required values) can be set in the builder. This means that a unit-test only has to set those properties of an object that are relevant for the specific test, not bothering with other properties. This makes the test smaller and more readable;
builders can be extended with all sorts of With… methods. For example a method like WithInvalidOrderDate(), which encapsulates an order date of 1990/1/1. This is more readable than .WithOrderDate(new DateTime(1990, 1, 1)) because the method explains to the reader of the test that the order has an invalid order date.

The downside of these builders is of course that they have to be coded, which is not the most fun part and decreases your productivity. Therefore I’ve started a T4 (Text Template Transformation Toolkit) template that generates a (partial) builder for each domain class. To extent a builder you can create a new partial class with the same name. Do not change the generated files because they will be overwritten by the generator.

Note that this is only the first version of the template. The next version will include better implementation for compiling the source files, support for other collections than ICollection only and a real implementation of singularizing the collection name.

Conclusion

I’ve used these builders in a several projects now and I must say that it really adds value to the readability and quality of the unit tests. The additional time required to create the builders is a small price for what you get in return, especially when you use a code generator.

Download

The sample project, including the T4 template, can be downloaded from this link: FluentTestDataBuilderExampleWithT4.zip.

UPDATE 27/7/2010: new version of the T4 template that includes: bugfixing, refactoring and use of EF’s PluralizationService: GenerateTestDataBuilders.tt v0.2.

WPF: Document binding with WebBrowser control

2010-07-01T12:47:40.0875442+02:00

The WebBrowser control in WPF is an excellent control for embedding a web browser in your WPF application. If you want to show a HTML document in your WPF app this control is also the way to go. However, in this current version of the control (up until WPF 4) it is not possible to use databinding because there is no dependency property available that you can use. The only possibility is invoking the method NavigateToString(string) somewhere in code.

Although this is working it doesn’t fit nicely in the currently popular MVVM pattern, in which everything (at least as much as possible) should be done using databinding. Moreover, when you want to display a browser control in every item of a listbox, a databinding solution is much simpler.

One of the solutions to solve this problem is to create a new custom control that derives from the WebBrowser control and adds a new dependency property to which you can databind your document. When the property changes you can simply invoke NavigateToString(string).

Another solution is using WPF’s ‘Attached Properties’ which can be used to extend existing controls with new properties and/or behaviors. In the code below I’ve created a new dependency property called Document. When this property changes the event handler DocumentPropertyChanged is executed that invokes the method NavigateToString(string) of the WebBrowser. This causes to control to display the document.

             1: 
            public
            static
            class WebBrowserExtentions

             2: {

             3: 
            public
            static
            readonly DependencyProperty
DocumentProperty =

             4:  DependencyProperty.RegisterAttached("Document", typeof(string), typeof(WebBrowserExtentions),

            new UIPropertyMetadata(null,
DocumentPropertyChanged));

5:

             6: 
            public
            static
            string GetDocument(DependencyObject
element)

             7:  {

             8: 
            return (string)element.GetValue(DocumentProperty);

             9:  }

10:

             11: 
            public
            static
            void SetDocument(DependencyObject
element, stringvalue)

             12:  {

             13:  element.SetValue(DocumentProperty, value);

             14:  }

15:

             16: 
            public
            static
            void DocumentPropertyChanged(DependencyObject
target, DependencyPropertyChangedEventArgs e)

             17:  {

             18:  WebBrowser browser = target as WebBrowser;

             19: 
            if (browser != null)

             20:  {

             21: 
            string document = e.NewValue asstring;

             22:  browser.NavigateToString(document);

             23:  }

             24:  }

             25: }

To use this new property in XAML you can simply do the following:

             1: "{Binding
HtmlDoc}" />

The WebBrowser is now bound to the property HtmlDoc of the provided DataContext (which will probably be a ViewModel in case of MVVM).

Question is though, is this better than creating a new control that is derived from WebBrowser? Personally I don’t think there is a real winner, but I do like the power of attached properties. One scenario that requires attached properties are controls that are sealed and thus inheritance is not possible.

LINQ To SQL: Exception when removing child object from Master-Detail relationship

2008-05-21T21:06:22.9123053+02:00

This week I ran into a quite unexpected exception that was thrown when I tried to remove an object from my LINQ ToSQL DataContext:

System.InvalidOperationException: An attempt was made to remove a relationship between a Master and a Detail. However, one of the relationship's foreign keys (Detail.MasterId) cannot be set to null..

The example in which this occurred isn't very exotic, just a Master table and a Detail table and a relationship with a foreign key MasterId in the Detail table (which doesn't allow null's). So it should be perfectly valid to remove a row from the Detail table and submit the changes.

Unfortunately there is a problem with this in LINQ To SQL, which is best explained by this post of Beth Massi. She states that to work around this problem you have to manually alter the dbml using an XML editor and add DeleteOnNull="true" to the association. And yes, this solves the problem.

Beth discovered this problem in VS2008 Beta 2 and it still exists in the final release. Let's hope the problem is solved with the upcoming SP1...

Text Template Transformation Toolkit (T4)

2008-04-19T17:01:48.2172201+02:00

InfoQ recently posted an article about a very interesting feature in Visual Studio 2008, Text Template Transformation Toolkit (also called T4). This toolkit is a standard feature of VS2008 and is all about generating text files based on templates. This toolkit is part of the Domain Specific Language Tools but it is certainly not limited to DSL's. One example of using T4 which I think is quite interesting is the generation of config files for different deployment scenario's, which can be found here.

Useful links:

MSDN: Generating Artifacts By Using Text Templates, the MSDN entry point with a list of many links about this topic

Text Template Transformation Toolkit by Oleg Sych, tutorial with a lot of examples and links

T4 Editor by Clarius, a free template editor which is very useful because template support in VS2008 is very limited

The T4 toolkit is also available for VS2005 in the VS2005 SDK v4 and the Guidance Automation Toolkit (GAT).

Deferred Execution behavior in LINQ providers

2008-04-07T19:21:43.6582158+02:00

LINQ uses a Deferred Execution model which means that nothing really happens until the results of the query are accessed, e.g. in a for(each)-loop. One of the advantages of this model is that you can compose complex queries in multiple steps to make them more readable. So from a execution point of view I expected that it should not matter whether you create the query in one complex statement, or in multiple smaller statements. But unfortunately that's not always the case...

To see the difference I've created an XML document and two simple queries that create Album objects when the where-clause matches.

The 'complex' query:

             1: var query1 = from a in albums.Descendants("Album")

             2: 
            where a.Element("Artist").Value
== "Radiohead"

             3: 
            select
            new Album

             4:  {

             5:  Artist = a.Element("Artist").Value,

             6:  Title = a.Element("Title").Value

             7:  };

The decomposed, more readable query:

             1: var query2 = from a in albums.Descendants("Album")

             2: 
            select
            new Album

             3:  {

             4:  Artist = a.Element("Artist").Value,

             5:  Title = a.Element("Title").Value

             6:  };

7:

             8: query2 = from a in query2

             9: 
            where a.Artist == "Radiohead"

             10: 
            select a;

The result from both queries is exactly the same, but they execute differently. What happens in Query2 is that first a list of all Album objects is created and then the ‘where’ part is evaluated over each object. This is different compared to Query1 in which an Album object is only created when it matches the ‘where’ part. So in this example Query1 is more efficient.

Is this what we should expect of deferred execution with LINQ? Yes, at least on implementations based on IEnumerable. What basically happens is that a chain of methods is created and executed in the same order as added to the query. This makes Query1: albums.Where(...).Select(...) and Query2: albums.Select(...).Where(...).

With implementations based on IQueryable, like LINQ To SQL, this is different in a way that it uses an Expression Tree to analyze/optimize the query. Using the same two queries with LINQ To SQL both queries execute exactly the same! Only one SQL statement is send to database and they both include the where-clause, which make them equally efficient.

In this example it isn't that much a problem but when more data is involved this is definitely something to be aware of when composing LINQ queries.

Vista Sidebar Notes disappeared

2007-11-07T22:42:13.5173852+01:00

Today I lost all my notes from the Notes gadget of the Vista Sidebar, which of course contained some very important stuff. After some research on the net it turns out that I'm not the only one who experienced this problem...

There is a way to get them back, but that depends on having the 'Previous Versions' turned on in Vista. If you have you can restore the settings.ini file that is in the following directory:

C:\Users\username\AppData\Local\Microsoft\Windows Sidebar

In this file all the settings of your active Sidebar gadgets are stored, including your notes.

In my case I could restore the file and after a restart of Sidebar.exe I got my notes back. So a good file to include in your back-up if you don't have the 'Previous Versions' option turned on.

.NET Framework source code

2007-10-06T12:16:45.8704456+02:00

With the release of Visual Studio 2008 a great new feature will be added: access to the source code of the .NET Framework libraries! In this post of Scott Guthrie he announces that VS2008 will support downloading the source code from the Internet so you are able to debug the .NET Framework libraries.

This is really a great thing because now we get more insight in how the .NET Framework is implemented and why some things might not work as expected. Up until now there is the possibility to use Reflector to explore the framework libraries, but this will be much easier as it supports the debug ability...

.NET 3.5: Types, Variables and Initializers

2007-08-15T21:58:16.6004225+02:00

In this post I will wrap up some of the smaller but very useful new features of .NET 3.5/C# 3.0. Other features of .NET 3.5 I've already covered are Extension Methods and Lambda Expressions.

Object and Collection Initializers
This feature makes it easier to create objects and assign values to the properties of the objects. Until .NET 3.5 you either had to write code that created an object and initialized each property of the object, or use a constructor that takes a list of parameters (which had to change of be overloaded when a property was added). The same is true for creating lists of object, calling the Add method for every new object. The following example shows that object and collection initializers makes this easier, without the use of parameter constructors:

In the example above the default constructor (without parameters) of Person will be used, witch will be executed before the property initialization. In case an alternative constructor is required, e.g. including a read only Id, the following syntax can be used:

Automatic Property get/set
The feature of automatic property takes away the burden of explicitly coding a private field and the get/set of a property, these are generated automatically by the compiler. The syntax is similar to the way of defining an abstract property or a property in an interface:

If necessary the get/set can be changed to limit accessibility, e.g. Name { get; private set; }. Of course the object initializers as explained above cannot be used outside the class anymore.

Implicitly Typed Local Variables and Arrays
With implicitly typed variables the type of the variable is not defined up front but inferred by the compiler using the initialization of the variable. Basically this looks the same as defining the variable as an object, but there is a huge difference: after initialization the type is fixed and cannot be changed, e.g. by assigning a value of another type.

            
              
              var
a 
              =
              
              
              5
              ; 
              //
               int
              
              
              var
b 
              =
              
              
              5.5
              ; 
              //
               double
              
              
              var
c 
              =
               a 
              +
               b; 
              //
               double
              
              
              var
d 
              =
              
              
              new
              
              
              int
              []
{ 
              1
              , 
              2
               }; 
              //
               int
array
              
              
              var
e 
              =
              
              
              new
              []
{ 
              1
              , 
              2
               }; 
              //
               implicitly
typed int array
              
              
              
              
              foreach
               (var
f 
              in
               e) Console.WriteLine(f);

Using the var keyword is only allowed for local variables, they cannot be used as class variables for example. Also, they have to be initialized when they are declared.

The usage of implicitly typed variables as aboveis a bit doubtful I think, in fact it does make the code less readable (although IntelliSense can help you out). In relation to LINQ and Anonymous Types it does make more sense though, but more on that later.

Anonymous Types
Anonymous Types are objects that are defined and created on the fly using object initializers. The compiler actually creates properties of the initializers which can be used later on. The name of the type remains unknown and cannot be used within the program.

            
              
              var
p1 
              =
              
              
              new
               {
Name 
              =
              
              
              "
              Person
I
              "
              , City 
              =
              
              
              "
              New
York
              "
               }; Console.WriteLine( 
              string
              .Format(
              "
              {0}
from {1}
              "
              ,
p1.Name, p1.City));

Like implicitly Typed Variables the usage of Anonymous Types in this context doesn't seem to be very useful because you really cannot do much with such an object. With LINQ the usage will be more frequent, e.g. to display a selection from an executed query:

            
              
              var
nyPersons 
              =
               from
q 
              in
               persons 
              where
               q.Address.City 
              ==
              
              
              "
              New
York
              "
               select 
              new
               {
Name 
              =
               q.Name,
City 
              =
               q.Address.City
}; 
              foreach
               (var
nyp 
              in
               nyPersons)
Console.WriteLine(
              string
              .Format(
              "
              {0}
from {1}
              "
              ,
nyp.Name, nyp.City));

The source code of these examples can be downloaded from here: TypesVariablesAndInitDemo.zip (,95 KB)