Query NoSQL From .Net Using Couchbase and LINQ

Historically, one of the biggest hurdles dealing with NoSQL solutions has been querying the data.  In particular, the learning curve for new developers trained in SQL has been very steep.

Couchbase recently helped to address this limitation with the release of N1QL (a SQL variant) in Couchbase Server 4.0.  N1QL vastly expands upon both the power of Couchbase as well as the ease with which developers can query the data.  Because the language is so similar to SQL, developers can quickly make the transition.  As an example, at CenterEdge we recently hired a new developer, and we had him up and working with N1QL queries within a day.

Now, Couchbase has announced the release of their Couchbase LINQ Provider for .Net.  The provider is available as a Nuget package which adds support to the existing SDK.

For .Net developers, using the LINQ provider for Couchbase makes the learning curve even shallower.  With just a few exceptions, developers trained to write queries using LINQ with NHibernate or Entity Framework can make the transition without learning any new syntax.  Plus, due to Couchbase’s JSON document data structure, the LINQ provider adds support for a variety of N1QL features that aren’t possible with more traditional table-based data stores.

So, if you’re a .Net developer interested in NoSQL and big data, I strongly encourage you to check it out.  It’s a big step in bringing the performance and scalability of NoSQL to the masses without the headaches.

Disclaimer:  I’m a extra excited about this release because it is an open source project I was able to contribute to.  You can see the source code on GitHub.

Couchbase and N1QL Security

As a developer at CenterEdge Software, I’ve had a lot of cause to use Couchbase as our NoSQL databasing platform over the last few years.  I’ve gotten really excited about the potential of the new Couchbase query language in Couchbase 4.0, called N1QL.  So excited that I’ve spent a lot of time contributing to the Linq2Couchbase library, which allows developers to use LINQ to transparently create N1QL queries.

In doing work with N1QL, I quickly realized that it may have some of the same security concerns as SQL.  In particular, N1QL injection could be a new surface area for attack in Couchbase 4.0.  That’s what I call the N1QL equivalent of SQL injection.  I found that while the risks are lower in N1QL than in SQL, there are still some areas that need to be addressed by application developers using Couchbase.

As a result, I did some research and recently wrote a guest post on N1QL security for Couchbase users.  It researches possible N1QL injection security concerns, then goes into how to protect your applications when using N1QL.


Windows Domain Account Lockout Mystery

In addition to development, I sometimes get saddled with some domain administration.  We recently encountered a strange mystery, where a user’s account was being locked out every day as soon as they booted up their computer.  They hadn’t even tried to login yet, but their account was being magically locked out.

After lots of research, all of the obvious solutions were excluded.  We finally tracked it down by turning on Kerberos logging on the client computer (http://support.microsoft.com/kb/262177).  We then found Event ID 14, stating “The password stored in Credential Manager is invalid“.  But there were no passwords stored in the Credential Manager!

At this point, we found this very helpful forum discussion that explains it: http://social.technet.microsoft.com/Forums/windows/en-US/e1ef04fa-6aea-47fe-9392-45929239bd68/securitykerberos-event-id-14-credential-manager-causes-system-to-login-to-network-with-invalid?forum=w7itprosecurity.

Apparently, your user account credentials can get saved to the SYSTEM (a.ka. local computer) account on the computer.  Once there, you can’t access it through any normal UI to remove it.  We think this probably had something to do with our RADIUS auth on the WiFi network, but we’re not sure.  Fortunately, the instructions in the post were spot on.

Download PsExec.exe from http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx and copy it to C:\Windows\System32 .

From a command prompt run:    psexec -i -s -d cmd.exe

From the new DOS window run:  rundll32 keymgr.dll,KRShowKeyMgr

The only additional note I would add is that you need to run the command prompt as an Administrator, if you have UAC enabled.

Correcting MVC 3 EditorFor Template Field Names When Using Collections

So, I recently ran into a problem with ASP.NET MVC 3 and editor templates when dealing with models that contain collections. If you handle the collection directly in the view, it works fine:

@For i as integer = 0 To Model.Locations.Count-1
    Dim index As Integer = i

        @Html.EditorFor(Function(model) model.Locations(i).Name)
        @Html.EditorFor(Function(model) model.Locations(i).Description)

In the above example, the fields will receive the names “Locations[0].Name” and “Locations[0].Description”, and so on for each field. This is correct, and will result in the model begin parsed correctly by the DefaultModelBinder if it’s a parameter on your POST action.

However, what if you want to make an editor template that always gets used for a specific collection type?  Let’s say, for example, that the collection in the model is of type LocationCollection, and you make an editor template named LocationCollection:

@ModelType LocationCollection
@For i as integer = 0 To Model.Count-1
    Dim index As Integer = i
        @Html.EditorFor(Function(model) model(i).Name)
        @Html.EditorFor(Function(model) model(i).Description)

Then you reference it in your view like this:

@Model.EditorFor(Function(model) model.Locations)

In this case, the fields will actually have the incorrect names on them, the will be named “Locations.[0].Name” and “Locations.[0].Description”.  Notice the extra period before the array index specifier.  With these field names, the model binder won’t recognize the fields when they come back in the post.

The solution to this problem is a little cumbersome, due to the way the field names are built.  First, the prefix is passed down to the editor template in ViewData.TemplateInfo.HtmlFieldPrefix.  However, this prefix is passed down WITHOUT the period.  The period is added as a delimiter by the ViewData.TemplateInfo.GetFullHtmlFieldName function, which is used by all of the editor helpers, like Html.TextBox.

This means that we actually can’t fix it in the editor template shown above.  Instead, we need TWO editor templates.  The first one for LocationCollection, and then a second one for Location:

@ModelType LocationCollection
@For i as integer = 0 To Model.Count-1
    Dim index As Integer = i
    @Html.EditorFor(Function(model) model(index))
@ModelType Location
End Code
    @Html.EditorFor(Function(model) model(i).Name)
    @Html.EditorFor(Function(model) model(i).Description)

By breaking this up into two editor templates, we can now correct the prefix in the second template.  The first editor template receives an HtmlFieldPrefix of “Locations”, so we can’t do anything with that.  However, the Location editor template receives an HtmlFieldPrefix of “Locations.[0]“, so all we need to do is remove the extra period and the problem is solved.  As you can see in my example above, I’m calling a method on TemplateInfo, FixCollectionPrefix.  This is a simple extension method which corrects prefix, which you just need implement somewhere in an imported namespace:

Public Sub FixCollectionPrefix(templateInfo As TemplateInfo)
Dim prefix As String = templateInfo.HtmlFieldPrefix
If Not String.IsNullOrEmpty(prefix) Then
templateInfo.HtmlFieldPrefix = Regex.Replace(prefix, ".(\[\d+\])$", "$1")
End If
End Sub

And that’s all there is to it. I certainly hope that the MVC team fixes this problem internally for their next release, but until then we’ll just have to keep working around it.

MVC 3 Unobtrusive AJAX Improvements

I started experimenting this week with MVC 3 and the new unobtrustive javascript frameworks, both for AJAX and validation. These are built around jQuery, and frankly are very nice pieces of work. There are, however, a few improvements that can be made to the Javascript files included in the distribution. You can obviously include these changes manually, it’s pretty simple, but hopefully one day Microsoft will roll these up in the next version.

I’m not sure how the unobtrusive stuff is licensed by Microsoft, so I won’t publish the modified code here until I’m certain it’s okay. I will, however, go over the modifications so that you can do them yourselves.

  1. Use fn.delegate instead of fn.live – By default, the unobtrusive AJAX library uses fn.live to monitor for anchor click and form submit events, and if the elements are set to operate using AJAX they are intercepted and an AJAX request is initiated.  However, this can be inefficient because the selector is actually being fired during during document.ready, even though it isn’t used for anything.  This means every single form, submit button, and link on your page is being checked for a data-ajax=”true” attribute during document.ready for no reason.  By using $(document).delegate(selector, eventType, fn) this is completely avoided. Note: This only works using jQuery 1.4.2 or later.
  2. Cache jQuery Objects – While this is a minor efficiency concern, there are several places with the event handlers where $(form) is called multiple times to convert an element into a jQuery object with a single member.  It is much more efficient to cache $(form) in a local variable and reuse it. You can also make finding the form more efficient using .closest(‘form’) instead of .parents(‘form’)[0].
  3. Inefficient Handling Of Null Callbacks – asyncRequest uses getFunction to get a callback function based on data attributes.  This is nifty system, because can take either a function name or inline javascript code.  However, if there is no function specified, it goes through the trouble of using Function.constructor to create an empty function.  This can be avoided by adding “if (!code) return $.noop;” to the beginning of getFunction.
  4. Global AJAX Events – Currently, there is no way to attach global AJAX event handlers that specifically handle only the MVC related AJAX events.  Your only options are to intercept ALL jQuery AJAX events, or to attach events to each specific AJAX request using data attributes.  However, there are definitely use cases for global event handlers that only affect MVC related AJAX events.  Therefore, I recommend updating the “asyncRequest” function to add information to the options passed to $.ajax.  In particular, during the $.extend(options, {…}) I am adding “mvcRequest: true” and “mvcTargetElement: element”.  Now you can attach global jQuery AJAX handlers using $(document).ajaxSuccess, etc, and then test to see if it is an MVC request.
  5. AJAX Updated Forms Validation – In the current implementation of the unobtrusive validation, it can’t handle forms added to the page via an AJAX request.  You have to manually call $.validate.unobtrusive.parse() after the page update before client-side validation resumes.  This can be addressed using a global handler, so long as #3 above is also implemented.  By using the attributes passed in, we restrict the processing so it doesn’t re-parse the entire document, only the updated region. The code for this is below:
$(document).ajaxSuccess(function (event, xhr, settings) {
    if (settings.mvcTargetElement) {
        $(settings.mvcTargetElement.getAttribute("data-ajax-update")).each(function () {

Please let me know if you see any other improvements that I’ve missed. I’m just beginning to dig through things, so I’ll keep updating as I find more as well.