Saturday, July 26, 2014 Eric Richards

Unfortunately, I've been busier than expected with work stuff, so my free time hasn't been very abundant. It's looking like I'm going to be porting the Pidgin Sip plugin's Sip stack to c#, since the Microsoft UCMA Sip library isn't really compatible with Office365. The existing options for libraries to do this don't seem to be very suited to handling the Lync-specific SIP extensions... :-(.

Hopefully, more Voronoi work should be coming up next week, so I'll have some material for a real post.

In the meantime, I did find some time to work on the new website today.

Comments are back, although I haven't imported the old Blogger comments to the new format yet. Those may turn up tomorrow if I get ambitious... Parsing them out of the Atom export is not hard, it's just kind of tedious.

I've also hooked up a new RSS/Atom feed. You should now be seeing a link in the top-right corner that will let you subscribe to any new posts via feedburner. Any existing subscriptions should also work now - I've got things setup so the old feed url should point to the new feed as well, thanks to Sean McAlinden's helpful post. Looking over the logs for the last week, it looks like some of you are actually subscribed to my site. I had thought that it was just the service that cross-posts my new posts to Facebook.


Saturday, July 19, 2014 Eric Richards

I've decided to move my blog off of Blogger. Blogger was great for getting started, but it has just become too painful to fight with going forward. I'm sick of fighting the Blogger templating to force it to display my content the way that I want it to. Blogger has a habit of absolutely mangling the html that I try to post. For posts consisting mostly of plain-text and images, this is not that big a problem, but I have to spend a ton of time trying to get code-heavy posts to render in a readable way. Over the last year, I've spent far more time tweaking my posted html to get it right than it has taken me in the last week to write a home-rolled blog engine for this new site to use, even counting extracting my existing content out of Blogger and converting it into a new format.

Beyond that, there are a number of things that I would like to do going forward that I simply cannot do easily with Blogger, but I can do very easily if I have control over the site. I'm far more comfortable writing html, javascript and server-side code than I will ever be at bending Blogger to my will.

I'm sure there will be some speedbumps, but I think this is a much better solution for me, and ultimately, for you, readers. I'm nowhere near finished, but I think I've got the essentials ironed out, so I'm going to go ahead with the switch-over.

Things that should still work:

  • Existing links to my page should still be valid, at least for content. I've put in quite a bit of effort trying to get the old-style Blogger urls to play nicely with my new site
  • All of the old content has been imported.
  • Images should still be fine, since those were all externally hosted and I haven't swapped those out yet. Eventually, I'd like to serve all my screenshots locally, so I can convert them to jpegs so pages will load faster (most are loss-less PNGs right now).

Things that are broken

  • Comments are disabled for the moment. Extracting the primary content from Blogger was enough of a pain, so I haven't bothered with the old comments yet. I also need to add server-side support for comments. If you have any questions about any of the tutorials in the meantime, at the top of each post is a link with my name that will allow you to send me an email. I'll try to get back to you as quickly as I can.
  • Rss feeds - From my analytics, I don't think anybody actually used the blog feed that Blogger provided, but I'm not going to bother with implementing one for the new site unless there is some demand or I have some spare time and get inspired.
  • Some of the older posts might look a little wonky. I'm going through them as I have time and making sure that the content I extracted from Blogger renders decently, but it is time-consuming. Particularly some of the oldest posts, when I was still using the online Blogger editor, before I standardized my workflow on Windows Live Writer, may be a little bit off.

Thanks for bearing with me. If you notice anything unusual, feel free to send me an email, it would be very helpful in pinpointing issues.


Monday, July 14, 2014 Eric Richards

For the last six or eight months, off and on, I’ve been trying to write some code that will create a Voronoi diagram from a set of random points, inspired by Amit Patel’s Polygonal Map Generation demo.  In the last week or so, I had a bit of a break-through, in that I finally managed to get an implementation of Fortune’s Algorithm put together that would actually work and generate the Voronoi edges and vertices correctly so that I could render them.  Since most of the existing implementations I’ve been trying to work from are either broken or an enormous pain in the ass to try to understand, I’m planning on writing up my implementation, once I’m finally happy with it.  I’ve still got a fair way to go, since right now I’m only outputting the graph edges and vertices successfully, and while that renders prettily, I haven’t gotten to the point that I have all of the really useful graph connectivity information being output yet.  Hopefully, if everything goes well, I’ll manage to finish that up by the end of the week, but this has proven to be a real bugger of a problem to solve.

As I mentioned, most of the implementations I’ve studied are either incomplete or broken.  Particularly, I haven’t found an example yet that correctly clips the edges of the generated Voronoi polygons to a rectangular area.  So long as all you care about is rendering the graph to the screen, this isn’t a big problem, since most 2D graphics libraries will happily draw lines that extend beyond the drawable area of the screen.  However, if you’re trying to subdivide a rectangular region into Voronoi polygons, its kind of nice to have your edges actually clipped to that region.  What I’m envisioning doing with this code eventually is using it to render 3D maps, subdivided into territories, but with an overlaid rectangular grid for moving units – think of the strategic map in Total War games or Lords of the Realm. 

After I got frustrated with trying to make the broken clipping code I was working from perform correctly, I trolled Google and came across the Cohen-Sutherland line-clipping algorithm.  This looked to be exactly what I needed, and wonder of wonders, the Wikipedia page actually featured a readable, reasonable example, rather than the obtuse academic pseudo-code you usually find there (see the Fortune’s Algorithm article…).  The only thing I would caution you about with the Wikipedia example is that it uses a coordinate system where the origin is the lower-left bounds of a rectangle as usually encountered in mathematics, rather than the upper-left origin we commonly use in computer graphics, so some tweaking is necessary.

The code for this example can be downloaded from my github repository, at https://github.com/ericrrichards/dx11.git.  The algorithm is included in the Algorithms project, while the example code is in the CohenSutherlandExample project.  The example code is a quick-and-dirty mess of GDI drawing code, so I’m going to focus on the algorithm code.

image

After clipping:

image


Thursday, July 10, 2014 Eric Richards

Tuesday was the anniversary of my first real post on this blog.  For the most part, I’ve tried to keep my content here on the technical side of things, but, what the hell, this is a good time to reflect on a year of blogging – what went well, what went poorly, and where I’m going from here. 

What I Meant to Accomplish (And What I actually Accomplished…)

Content

I restarted this blog about a year ago to document my attempt at learning DirectX 11, using Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0.  In my day job, I mostly code ASP.NET and Winforms applications using C#, so I decided to convert the C++ examples from Mr.Luna’s book into C#, using SlimDX as my managed DirectX wrapper.  SlimDX appeared to me to be slightly more mature than its main competitor, SharpDX, as its documentation was a little more complete, and it had been out in the wild a little bit longer, so there was a bit more third-party information (StackOverflow, other blogs, GameDev.net forum postings, etc.) on it.  I suppose I could have also gone with XNA, although Microsoft appears to have abandoned any new development on it (Last release 9/16/2010…), and I felt like SlimDX’s simple wrapper around the DirectX library would be easier to translate than shoehorning into the XNA model, not to mention wrassling with the XNA Content Pipeline.

My initial goal was to work through the book, converting each of the examples presented and then blogging about the process.  Except for the chapters on Compute Shaders and Quaternions (which I have yet to tackle, mostly because the examples are not terribly interesting), I completed that goal by the middle of November of last year.  From there, I started incorporating elements from Carl Granberg’s Programming an RTS Game with Direct3D into the terrain rendering code that Luna’s book presented, as well as dabbling in integrating Direct2D and SpriteTextRenderer to handle 2D drawing.

After that, my intention was to start working my way through Ian Millington’s book, Game Physics Engine Development.  This is about where I ran out of steam.  Between the hassle of trying to reconcile the code examples from this book, which were based on OpenGL and somewhat less self-contained than what I had been working on previously, various issues in my personal and work life, and the general malaise of an especially cold, dark winter here in New England, my impetus to work on my side projects faded away spectacularly.  If any of you followed this regularly, I’m sure you’ve noticed that it’s been almost four months since I’ve posted anything new, and before that there was another dry spell of more than a month. 

With the arrival of summer and a move that reduces both the strain on my finances and the number of hours per day I spend driving back and forth to work considerably, I’ve found that my mood has improved by leaps and bounds, and I have the extra energy to expend on coding outside of work again, finally.  Right now, I’ve got a number of new things I’m working through, and a number of ideas in the pipeline that should be making their way up here in the near future.

Sharing

In addition to learning things myself, another of my goals was to share what I learned with anyone else out there interested in graphics and game programming.  Putting this content up here was the first step, but the internet is littered with interesting information buried and inaccessible because the GoogleBots either can’t find it or deem it unimportant.  I don’t pretend to know anything about SEO, and I’m not sure its something I really want to get involved in – everything I’ve read seems to indicate that you either need to get lucky and go viral, or else spend a bunch of time or money doing vaguely unethical things, like spamming links or hiring a shady outfit in the Ukraine or China to do it for you. 

So, my limited efforts at promotion concentrated on GameDev.net, Facebook, Twitter, and HackerNews. 

  • GameDev.net was probably the most effective.  The main source of views came from cross-posting each post I made here on my developer journal there (http://www.gamedev.net/blog/1703-richards-software-ramblings/).  A couple of these posts got picked up by the admins and made the front page, under the Featured Developer Journals section, which resulted in a pretty big boost in views.  I also added links to here on my forum signature, which I don’t believe amounted to much.  To some extent, I also trolled the Beginner and DirectX forums, suggesting that anybody that was having a problem I had covered check out the relevant post.  I hope that this was somewhat helpful, and not just spammy…
  • Facebook was probably not worth bothering with…  Generally my friends on Facebook are split between people I knew in high school, family members, my fraternity brothers from college, and other Dartmouth people who trend towards being law students, med students, teachers, or I-bankers.  It’s not exactly a fertile demographic for tutorials on 3D graphics programming.  Now, if I was writing Top-X lists of Marvel characters, I might do better there, but that’s the nature of Facebook…
  • Twitter is also kind of a non-starter.  Given that I have a grand total of 11 followers, and I don’t really believe in the whole idea of Twitter, that’s probably not surprising.
  • HackerNews was very hit or miss.  Most of the time I would post a link, and it would languish on the back pages forever.  Once in a while, however, something would get some bizarre traction with one of the Hacker News aggregators (usually Feedly), and I’d see a big spike in views for a post.  I have no idea what the rhyme or reason for this is; my best guess is that I happened to post at a particularly dead time, and my link stayed near the top of the newest links page for longer than normal.

However, the vast majority of people who made it to my site came from Google searches.  I’m not sure how, but my site has come to rank pretty highly for certain Google searches.  Some examples, as of today:

Some Charts…

image

Oddly enough, the biggest growth in traffic came after I ran out of steam and stopped posting.  I wonder if this was a missed opportunity to continue growing.

image

Kind of an interesting split in the most popular pages on the site.  Mostly, this is my more interesting and more advanced content, although the two most basic examples on the site are also represented.  Also, my tutorial index page is doing pretty well.

Finances

Making money isn’t really my goal with this site, which is a good thing, since it certainly has not been a mint.  Fortunately, maintaining the site really doesn’t cost me anything: I use Blogger for hosting, which is free, and my GoDaddy registration only costs $10 for the year.

Hopefully, the banner ads I’ve got on the site are not that obnoxious.  Probably most of you who would see this use AdBlock anyway.  They have more than doubled my monetary investment in the site over the past year – as of today, I’ve earned just over $25 from AdSense.  At this rate, I’ll hit the minimum payout threshold in another three years, haha.

If I were to take a wild guess at the number of hours I’ve spent coding and writing up these posts over the last year, and then calculating an hourly rate based on my AdSense revenue, I’m probably earning about a nickel an hour… Even mowing grass or stacking firewood would be orders of magnitude more lucrative, but whatever – I’ve arguably learned more about programming and computer science in a year of porting C++ to C# and fighting with shaders than I did during my courses for my Comp Sci minor.  Even better, kicking around the dusty corners of the .NET library to match up with C++ constructs has broadened my knowledge of what is available, and spending so much time in Visual Studio coding and debugging has taught me all sorts of tricks that I can use in the day job too.

What’s Next?

In a perfect world, I’d continue to churn out high-quality, interesting content on a regular basis, and this site would become a go-to resource for SlimDX and general C# game programming, similar to the LazyFoo or RasterTek tutorial series.  I’ll settle for just getting back to a more consistent posting schedule, though.  There are so many interesting topics to consider, and both coding and then explaining what I’ve done is the best way I have found so far to cement my understanding of algorithms.

As I mentioned, I have a bunch of stuff in the pipeline that I’m hoping to finish and write up in the near future, so hopefully there will be some new content here shortly.

I’ve been toying with the idea of writing a book, since SlimDX does not appear to be very well covered in print (At present, Amazon only lists one title).

Anyway, thanks for reading over the past year, and I hope that this site has been useful.  Here’s to an even better second year of http://www.richardssoftware.net/!


Thursday, March 20, 2014 Eric Richards

At my day job, we are working on a large enterprise system.  One of the feature areas of this product is an array of different charts of performance metrics.  All of these charts accept a common set of parameters, and generate some json that is used by our client side to render a chart for the requested metric.

At the moment, we are using a standard MVC controller to define the actions necessary to calculate and collate these stats.  The other day, we were kicking around the idea of how we would be able to add new charts in for a client, after they already have the product installed.  With the current design we are using, there isn’t really an easy way to do that without dropping in the updated build that has the updated controller.  That’s not really an ideal situation, since reinstalling and reconfiguring the application is kind of a heavy-handed approach when all we want to do is add a couple of new charts in, without impacting the rest of the application.  It also doesn’t give us many options if our customer, for whatever reason, wants to disable certain charts.

Probably, what we’ll end up doing, is using the database to define the chart calculation SQL, either using by storing the SQL statements to retrieve the metrics in a table, or via stored procedures.  I’ll admit, I’m not crazy about this approach, since I’ve worked on a couple of other products that used this method, and I have some slight PTSD from trying to troubleshoot convoluted SQL that was poorly written and not checked into source control other than in the master database creation script.  With a sane process, this will probably be an effective option; one of the reasons we are likely going to need to go this route is that we are developing parallel .NET and Java versions, due to API restrictions of the underlying technology stacks that we are targeting, while trying to use a common UI and database.

This did get me thinking about how it would be possible to dynamically populate the actions of an MVC controller.  I had come across some posts on CSScript, which is a library which allows you to run arbitrary C# code interpretively.  Depending on how you use it, you can load up entire classes, methods or even statements, and invoke them at run-time, without the script code being compiled and embedded in your .dll.  So, theoretically, it should be possible to define the controller action that calculates the metrics for a given chart in a CSScript file, load that up, execute it using the CSScript interpreter, passing any parameters needed in, and return the results.

The other piece of this is figuring out how to get an MVC controller to accept action routes that are not explicitly defined for it.  Fortunately, with the latest version of MVC, you are able to tweak the default behavior of the controller to a considerable degree, if you are willing to delve into overriding some of the virtual methods the the Controller class gives you access to.  So, after a little digging, I was able to figure out a slightly crazy way to wrestle MVC into doing what I wanted it to do – accepting dynamic controller actions, executing a scripted action, and then returning the result to the webpage.

This is a very quick and dirty example, really just a proof of concept prototype, so I make no promises as to its performance or robustness.  But, it was cool to figure out that such a thing was possible, and I figured that I would share it as a jumping-off point in case anyone else encounters a similar crazy requirement.  The code is available on my GitHub repository, at https://github.com/ericrrichards/MVCScriptedController.git.

As a simple example, let’s assume that we know that we are going to have to support controller actions with a signature like this:

ActionResult Action( string i )

An example implementation of an action like this would look like this (from Scripts/Reports/Foo.cs in the project):

public ActionResult Foo(string i) {
    return new JsonResult() {
        Data = "Foo - " + i,
        JsonRequestBehavior = JsonRequestBehavior.AllowGet
    };
}

We’ll start with an empty MVC5 project, and then add a new empty Controller to the project.  Deleting the auto-generated Index action, that leaves us with this:

using System.Web.Mvc;

namespace ScriptedController.Controllers {
    public class ReportController : Controller {

    }
}

The Controller class offers us a virtual method, void HandleUnknownAction(string actionName), which is fired when a request is routed to the controller and the requested action does not match any of the defined controller actions.  By default, this method just throws an HTTPException with a 404 error code.  Interestingly enough, there must be something else in the Controller class which catches this error, because if you were to actually hit a non-existent controller action without overriding HandleUnknownAction, you’ll get a 200 response with no content.  However, we can override this method to allow us to do something else, which in our case will be to look for a dynamically loaded action method delegate and execute that instead.

To support this, we are going to add a delegate to match the action signature that we are expecting, and a dictionary to map action names to these delegates to our controller. We’ll make this dictionary static, so that it can be shared between all instances of the controller that we are creating – this will save us some time on each request, since MVC creates a new Controller instance for each incoming request, so that we would otherwise need to load up our dynamic Actions for every request. 

private volatile static Dictionary<string, MvcAction> _actions;
private delegate ActionResult MvcAction(string i);

We can now overload the HandleUnknownAction method, so that it checks this dictionary for an action which matches the passed actionName.  If the action is found, we will execute it, and write the result to the http response, using the ActionResult.ExecuteResult() method.  Otherwise, we will attempt to invoke the base HandleUnknownAction method.  For whatever reason, when we do this, the resulting exception is not caught, so we need to handle that and set the response code to 404.

protected override void HandleUnknownAction(string actionName) {
    if (_actions.ContainsKey(actionName)) {
        _actions[actionName](HttpContext.Request.Params["i"]).ExecuteResult(ControllerContext);
    } else {
        try {
            base.HandleUnknownAction(actionName);
        } catch (Exception ex) {
            HttpContext.Response.StatusCode = 404;
        }
    }
}

Getting the parameters to the action is somewhat hacky.  We can access any of the parameters passed along with the HTTP request by using the HTTPContext.Request.Params dictionary.  In this situation, where we know that all of the dynamic actions for this controller will share a common set of parameters, we can hard-code the parameters that we extract – at the moment, I’m not totally sure how I would handle a more general case, so that bears some thought.

At this point, we have a controller that can handle actions requested of it that are not defined explicitly in the class, assuming that they are contained in its dictionary of dynamic actions.  The next step is figuring out how to populate that dictionary with the dynamic actions.  For that, we need to use CSScript.  We are going to add another method to our controller, named EnsureActionsLoaded(), which will handle this for us. 

What this method will do, is check whether the actions dictionary has already been loaded, and if not, load the actions from a directory which contains a series of script files, where each file contains a single action method, with the filename matching the action name.  We’ll use CSScript to dynamically load each method as a delegate, which we can then push into our dictionary. 

private void EnsureActionsLoaded() {
    if (_actions != null) {
        return;
    }
    lock (SyncRoot) {
        if (_actions == null) {
            _actions = new Dictionary<string, MvcAction>();
            var path = Server.MapPath(ScriptsDirectory);

            foreach (var file in Directory.GetFiles(path)) {
                try {
                    var action = CSScript.Evaluator.LoadDelegate<MvcAction>(System.IO.File.ReadAllText(file));
                    var actionName = Path.GetFileNameWithoutExtension(file);
                    if (!string.IsNullOrEmpty(actionName)) {
                        _actions[actionName] = action;
                    }
                } catch (Exception ex) {
                    Console.WriteLine(ex.Message);
                }
            }
        }
    }
}

If you have dealt with the singleton pattern in C#, you may have noticed that we are using a similar double-check locking method to determine if the action dictionary has already been loaded.  Since the dictionary is static, and because of the multi-threaded nature of web applications, it is possible that two or more simultaneous requests will arrive at once, and if both attempt to load the actions dictionary at the same time, they have the potential to stomp on each other.

Since we are using the Server.MapPath() function to determine the location of our script directory, we need to also override the Initialize() method of the controller to call our EnsureActionsLoaded method.  Ideally, we could do this in the constructor, but at that point, the Server variable is not yet initialized.

protected override void Initialize(RequestContext requestContext) {
    base.Initialize(requestContext);
    EnsureActionsLoaded();
}

At this point, we have a working controller which can load CSScript files as controller actions, and execute them.  There are a couple of simple examples of these scripts in the Scripts/Report folder of the project, like the following Fizz.cs:

using System.Web.Mvc;
public ActionResult Fizz(string i) {
    return new JsonResult() {
        Data = "Fizz - " + i,
        JsonRequestBehavior = JsonRequestBehavior.AllowGet
    };
}

One thing to note is that you will either need to use fully-qualified type names in your scripts, or else include the necessary using statements for CSScript to properly parse and compile your scripts.  I haven’t tried anything more complicated yet using assemblies that are not referenced in the project itself, so be aware that there may be some gotchas there.

Altogether, our dynamic controller looks like this:

using System;
using System.Collections.Generic;
using System.Web.Mvc;
using System.IO;

namespace ScriptedController.Controllers {
    using System.Web.Routing;

    using CSScriptLibrary;

    public class ReportController : Controller {
        private const string ScriptsDirectory = "~/Scripts/Reports";

        private static readonly object SyncRoot = new object();
        private volatile static Dictionary<string, MvcAction> _actions;
        private delegate ActionResult MvcAction(string i);

        protected override void Initialize(RequestContext requestContext) {
            base.Initialize(requestContext);
            EnsureActionsLoaded();
        }

        private void EnsureActionsLoaded() {
            if (_actions != null) {
                return;
            }
            lock (SyncRoot) {
                if (_actions == null) {
                    _actions = new Dictionary<string, MvcAction>();
                    var path = Server.MapPath(ScriptsDirectory);

                    foreach (var file in Directory.GetFiles(path)) {
                        try {
                            var action = CSScript.Evaluator.LoadDelegate<MvcAction>(System.IO.File.ReadAllText(file));
                            var actionName = Path.GetFileNameWithoutExtension(file);
                            if (!string.IsNullOrEmpty(actionName)) {
                                _actions[actionName] = action;
                            }
                        } catch (Exception ex) {
                            Console.WriteLine(ex.Message);
                        }
                    }
                }
            }
        }
        
        protected override void HandleUnknownAction(string actionName) {
            if (_actions.ContainsKey(actionName)) {
                _actions[actionName](HttpContext.Request.Params["i"]).ExecuteResult(ControllerContext);
            } else {
                try {
                    base.HandleUnknownAction(actionName);
                } catch (Exception ex) {
                    HttpContext.Response.StatusCode = 404;
                }
            }
        }
    }
}

If you run the project, you should be able to hit both /Report/Foo and /Report/Fizz, and see results like the below:

image

image

Attempting to hit an unregistered controller action will give you the lovely ASP.NET 404 error page:

image

I’m not sure if I will ever get a chance to actually use this, but it was a fun little experiment.  Hopefully if any of you find yourself with a similar problem, this will give you a starting point to work from.