slaingod

I know this is sacrilegious, but there is some serious lameness going on in ActiveRecord I’ve dealt with lately. Maybe I’m drawing outside of the Rails lines (going off the tracks?), but ActiveRecord seems to go out of its way to make things a pain in the ass.

AR::Base#sanitize_sql being a ‘protected’ method has always been a burr in people’s sides. This means you can’t call it yourself on your own piece of SQL. Presumably it is done this way so people HAVE to do it the Rails way, whether that means duplicating a bunch of code, or taking a lot more time for a one-off project, etc.

Currently I’m working a report generator for Flex, where the Flex app handles the SQL generation and passes back an XML version of the sql options like:

<query>
    <name>
    </name>
    <sql>
        <select>
            <![CDATA[Date(created_at) as date]]>
        </select>
        <select>
            <![CDATA[count(id) as total]]>
        </select>
        <from>
        </from>
        <conditions>
        </conditions>
        <group>
            <![CDATA[date]]>
        </group>
        <having>
            <![CDATA[date >= :start_date and date <= :end_date]]>
        </having>
    </sql>
</query>      

Now we can argue all day about the best way to do this, but the reality is that only Admin authenticated people are going to see these reports, so the fact that someone could send arbitrary sql against the database is outweighed by the Flex-ibility of being able to dynamically adjust the query values in Flex without having to create a custom server-side method for each report. It is easier in my case to let more readily available/cheaper Flex programmers handle this than more expensive Ruby coders.

Getting this accomplished in Rails led to 5 workarounds in the code:

  def replace_named_bind_variables_no_quotes(statement, bind_vars) #:nodoc:
    statement.gsub!(/:(\w+)/) do
      match = $1.to_sym
      if bind_vars.include?(match)
        bind_vars[match]
      else
        raise ActiveRecord::PreparedStatementInvalid, "missing value for :#{match} in #{statement}"
      end
    end
  end

  def query

    # from_xml puts it in something like { queries => {query => [{name, sql}, {name,sql}....] }}
    queries = Hash.from_xml(params[:queries])['queries']['query']
#     logger.dbg queries.inspect

        # generate the report structure
        report = []
        queries.each { |data|

            query = data['sql']
            logger.dbg query.inspect

            # Now we need to get around a bunch of ActiveRecord lameness....
            # Presumably it is done this way to satisy someone's idea of 'how you should do things'
            # rather than, 'let's help them do it, no matter how they want to get it done'

            # first, AR doesn't do bind variables for anything but conditions...
            replace_named_bind_variables_no_quotes(query['group'], params) if query['group']
            replace_named_bind_variables_no_quotes(query['having'], params) if(query['having'])

            # second, we need to join the select clauses, as :select doesn't accept an array...   
            query['select'] = query['select'].join(', ')

            # third, AR doesn't support a separate HAVING clause, you have to attach it to GROUP BY
            if query['having'] and query['group'] # you always have both...
                query['group'] = query['group'] + " HAVING " + query['having']
                query.delete('having')
            end

            # fourth, we need to intern the keys so that they pass 'inspection' by AR
            interned_query = {}
            query.each { |key, value|
                interned_query[key.intern] = value if(value != nil) # AR doesnt like :conditions => nil either...
            }
            logger.dbg interned_query.inspect

            # fifth, we need to use Creative instead of just ActiveRecord::Base because there is a bug/weirdness in reset_table_name 
            # where it can't find the abstract_class
            report << [data['name'], Creative.find(:all, interned_query)]
        }

    # output the results xml
        str = ''
        xml = Builder::XmlMarkup.new(:target => str, :indent => 1)

        xml.result {
            report.each { |query|
                xml.query {
                    keys = []
                    xml.name query[0]
                    xml.cols {
                        exemplar = query[1].first
                        exemplar.attributes.each { |key, value|
                            xml.col key
                            keys << key
                        }
                    }
                    xml.rows {
                        query[1].each {  |row|
                            xml.r {
                                keys.each { |key|
                                    xml.v row.attributes_before_type_cast[key]
                                }    
                            }
                        }
                    }
                }
            }
        }
        render :xml => str

  end

So on a little side project I am working on, I realized that I needed to roll back to .NET Compact Framework (NETCF) v1.0 from a newer version. Why, you might ask? Well, it seems that even on my fairly recent Windows Mobile 5 XV6700 HTC Apache/Mogul whatever (only 2 years old, hey now), that not even NETCF v2 is available. Now you can always just install NETCF 2 or 3.5, but those take up 5+ MB of precious space on your device. Even worse, when you start even the simplest app, your RAM usage can balloon to 3-4 MB with just a few simple controls.

Using the built-in NETCF (v1.0-ish) can help reduce those issues dramatically.

I was never able to figure out how to create a new NETCF v1.0 project in Visual Studio 2005, so I went back and installed VS 2003. Created the project, then opened it in VS2005 and added all of my old files. I need to use VS2005 because I am using Vista x64 and the device emulator in VS2003 doesn’t work in that environment it seems (drivers wouldn’t load). It is also my understanding that you can’t move forward to VS2008 either, that you are forced to upgrade the framework version.

So once I got my files included, I tried to compile. Ooops! Quite a few problems I had to fix popped up.

• The shorthand for property get/set (get; set;) didn’t work. I had to go back and redo all of my properties that used this. There is still a ‘prop<tab><tab>’ helper though.
• My resources were all messed up. I had to basically rebuild my forms from scratch, then copy in all of the stuff from the newer NETCF versions. I took the opportunity to simply move a lot of my resources into a skin folder and just load them at runtime with FileStream and Bitmap.
• I had a splash page that was just another form, and used TopMost to make sure it was on top. This isn’t available in NETCF v1, and it really isn’t clear to me at all how control paint ordering occurs by default. BringToFront/SendToBack didn’t really do much for me, and the order of creation of the controls didn’t help either. There is supposed to be a way to do this with SetWindowPos.
• List<string>-style templates aren’t available, or anything from System.Collections.Generic. You need to use ArrayList or similar instead, and some occasional casting from object.
• The MenuItem.Tag(and Tag from other controls) is missing, which is where you store your user data for menu controls. For instance you might create a list view of files, and each file would have a Tag object containing its FileInfo object. You have to come up with another solution here, like keeping track of the order/array of MenuItems, unless I figure something else out.
• Enum.Parse isn’t available.

It seems a lot of these problems can be solved by using the OpenNETCF Smartphone Device Framework v1.4. For instance EnumEx.Parse is available in that framework addition.

So I’m new to C#, but I have to admit that the ‘simple’ examples out there really aren’t simple and they really don’t cover what I would consider to be the most common use for events: Communication between forms and controls without having to resort to directly linking one class to another. I’ve been using Cairngorn Events/Delegates for awhile now in Flex/ActionScript3, so I’m not coming at it from a complete novice standpoint. But it seems every example I could find only showed how to raise and use events within the creator of the event. None of them showed how to use events to decouple, or reduce the linkage and increase the usability of your code.

So here goes: Imagine if you will, that you have several forms on your application and you want to allow the user to exit from any of them, while still doing that last minute clean up like saving any data or config, etc. ( I realize there is a way to do this using the Application event handler, but this technique applies all over the place, and this example is easy to understand.)

So first you want to create your event class. I put mine in a handy folder called Events in my C# project.

using System;

namespace Demo.Events
{
    // Declare a delegate for an event.  
    public delegate void ExitAppHandler(object sender);

    // Declare an event class. 
    class AppExitEvent
    {

        public static event ExitAppHandler evtExitApp;

        // This is called to fire the event. 
        public static void OnExitEvent()
        {
            // ok so this calls the handlers we have +='d to listen the event
            if (evtExitApp != null)
                evtExitApp(null);
        }
    }
}

OK, so this sets up a separate event class that has a static handler variable and a static method. This is the glue between our classes. Now we can reuse a form that calls this event, without having to worry about any other code.

Now we want to listen to this in our forms that want to do something when this event is triggered.

namespace Demo {

    public partial class Demo : Form
    {

        public Demo()
        {
            Events.AppExitEvent.evtExitApp += new Events.ExitAppHandler(this.ExitEventHandler);
            InitializeComponent();
        }

        // An event handler. 
        private void ExitEventHandler(object sender)
        {
            // TODO: Do cleanup/save/etc
            this.Close();
        }
       }
}

This could be your main application form for instance, with a lot of other stuff going on in this class, like adding controls, etc. Multiple forms that need to respond to the exit event would simply do the same thing: Add Events.AppExitEvent.evtExitApp += new Events.ExitAppHandler(this.ExitEventHandler); to their class/form, along with an ExitEventHandler method.

Now all we need to do is show another form/control/class calling this as well. In my case, this is a custom toolbar, or a sub form’s right click menu:

namespace Demo
{

    public partial class Caller : UserControl
    {

        public Caller()
        {
                      // If you wanted to have this form respond to the exit event as well, you could simply uncomment this
                     // Events.AppExitEvent.evtExitApp += new Events.ExitAppHandler(this.ExitEventHandler);
                 InitializeComponent();
        }

                //  this is in response to a button click NOT SHOWN
        private void pbExit_Click(object sender, EventArgs e)
        {
            Events.AppExitEvent.OnExitEvent();
        }

    }
}

And finally, we show how to call this static event: Events.AppExitEvent.OnExitEvent();

This is called in response to a button click, that button not shown in this example, but you should hopefully get the idea. I’ve also shown commented out, that this other form could also be a consumer of the same exit event if you wanted.

That does it! I needed another similar event to occur when I minimized the app, and when I wanted to open a popup form from a number of different locations, so this technique is very useful.

Obviously, the above example doesn’t really show how to pass additional info with the event, like mouse coordinates or other data.

To do that you would need to create a class derived from EventArgs (CustomEventArgs let’s say) that held your custom data, and change the declarations to add in your custom event args. The existing examples out there show this aspect pretty well.

Here is a little AutoIt script I found while trolling their forums that will put your monitor into sleep mode at the push of a button.  Both the script and a compiled EXE version are included, so you don’t have to install AutoIt to use. I put a shortcut to the script into my QuickLaunch bar, then changed the icon to the power button icon, and now whenever I am done with my computer for the time being I can press this button and my monitor goes off right away!  There are plenty of other shareware, etc. tools out there to do this, but this one is so simple, it’s basically a single line of code. I grabbed this off of the AutoIt forums, and commented out some of the existing code.  That code originally would turn off the monitor after 10 seconds of non-use (which is usually useful in a Network Operations Center but not much else).

Download the script!

Working in the ‘media sharing’ space, I’ve come across the need for a SWF to FLV application (swf2flv).  There are a number of products out there, and some oldish Python scripts, but none of them really satisfied my needs. Some of the products out there are by Sothink and Scout.  I needed something that could be called from a command line/console, and Scout does have a version that does that, though it costs $1000ish for a single server license. But it didn’t have one key feature I needed, so I started to look into what it would take to build my own.  Hence Flivver (think FLV, not Aldous Huxley).

Here are the basic things that Flivver needed, with the one extra feature that I haven’t seen in any other SWF2FLV app:

• Process a SWF without any human interaction from a server: This won’t work with swf’s that require button presses, but for my needs I needed to take an audio/visual playlist created by a user and have it converted into something that user could possibly upload to YouTube.
• Video/Visual capture: Capture everything that happened in the FLV, including bitmap manipulations, and text, effects and transitions.  While it wouldn’t be too hard to append a bunch of video files together, once you start adding affects like ‘Black & White’ to the video, you have to really consider how you are going to integrate you playlist with some pretty serious Adobe Premiere/iMovie type application to replicate it in a standalone video file.
• Audio Capture: In the same way as video, you need to be able to capture the audio as it occurs in the playlist, with transitions and effects.
• Be able to assign a playlist id to the SWF, so it can load the playlist from the internet. One of the key things missing in all of the SWF2FLV and SWF2AVI apps is the ability to set flashvars or otherwise interact with the SWF in any meaningful way.  Specifically for my needs, I have to load the playlist from a web server application that stores the playlists created by the users.  As far as all of the other server/console style converters, they don’t support any way of interacting with the swf being processed.

So as any good hacker would, I started searching google for info on how to do the various pieces: Video Capture, Audio Capture, Hosting Flash in an Application, & Talking to Flash from a Hosting Application. I found a lot of good information, but it is all about weeding out the bad.  The first decision I made was that I was going to have to use the Windows environment for the application, simply because I am more familiar with it as far as GUI programming goes, and there was a lot more information on screen capture out there for it, as well as using video card acceleration to speed up the process. I had done some work with xine back in the day (dealing with the Via EPIA Hardware a few years ago, though nothing official), and just felt that for my needs the effort would have been to great to get a working version on linux. Also, in general our production servers didn’t have X11 running or sound cards anyway, so we needed our own GUI server for everything to work.

So then the question was whether to do it in C++ or C#.  Although there are more examples in C++, I decided to go with C#, because I wanted to try something new. So that decision helped me narrow down my google searches.

First I needed to find C# versions of the DirectShow SDK, with examples, as I really didn’t want to learn all the ins and outs of dealing with DirectShow filters & COM.  I finally came across the DirectX.Capture demo at CodeProject, which had about 50% of what I needed, and was the scariest part for me.  The next piece I needed was a DirectShow Filter that did screen capture. I found a couple of those: UScreenCapture and VH Screen Capture filters. VH Screen Capture has a few more options I believe, but for now I am using UScreenCapture. So another 25% of the project down. Next I needed a way to capture the audio that was being played through the sound card.  This is easily accomplished by setting the Audio source to ‘Stereo Mix’, and capturing with a reasonable volume at 44khz, 16 bit stereo. (NOTE TO LAPTOP/MACBOOK BOOT CAMP USERS: You may need to download some hacked SigmaTel HD Audio drivers or make some changes to your driver installation files to get ‘Stereo Mix’ to be an option for your Volume Mixer. Do some googling for “Sigmatel MacBook Stereo Mix”.) Finally, I needed a way to embed Flash into my C# application in a way that allowed me to call Flash methods. I found the instructions to this at this site. Note: that if you are using Visual Studio 2005 or newer, you will need to download a Flash ActiveX COM definition file as Microsoft stopped shipping one after VS2k3. You can find information here.

One challenge I wasn’t able to solve is having the output be in FLV format directly form the C# app. I specifically wanted to output On2 VP6 format rather than Sorenson, as it is much better quality at the same or even lower bitrates.  I couldn’t find any free DirectShow filters for this, and the one’s from On2 cost $1000 per server again.  The way to work around this is to use MPlayer’s MEncoder with the VP6.2 codec/filter installed (which even works on Linux although you need to create the settings vcf file on Windows). So  Flivver is a bit of a misnomer, since it actually outputs AVI in xvid/mp3 formats, which is then converted by mencoder.

Now that I had all of my major technological hurdles taken care of, it was just a matter of integrating them into a single application. The basics pieces were:

• Add command line handling to the application so it could have information passed in from the server.  Specifically, this app is called by a Ruby on Rails library.  The app receives a post from the production Rails servers telling it which playlist needs to be processed, and this is then passed in on the command line.
  
• Get rid of any of the DirectX.Capture settings and UI that we don’t need.  We know which specific filters we want to use, so we don’t need to deal with all of the selection menus, etc.
• Add some timing handling.  There is no way I could tell to discover automatically the state of the SWF being loaded into the ActiveX control. The Flash onReadyStateChange didn’t really give me anything useful.  Because I had access to the code for the SWF I wished to capture, I was able to add some communication between the the SWF and the C# app, so that things could be coordinated, like when to start capturing and when to end.

That’s basically it.  I have created a SourceForge project for Flivver, though there is nothing there at the moment.  I am still finishing up some of the integration details, and changing things from hard-coded stuff I needed for my particular usage to more flexible settings/command line based options.  While there may be limited usage for something like this, I think it is still a cool little project and maybe will help someone else who has to solve a similar challenge. I’ve picked an MIT license for release, though I still need to go through all of the code I cribbed off of and see what the details for those licenses are, so that may change.  At the very least you can use my instructions here to get going on your own if need be. 

Sadly, flivver.com is a parked domain as a misspelling of ‘flower’.

And finally, thanks to my peeps at UGENmedia for paying my bills and encouraging me to put this out there for others.

Let me know what you think!

So I don't really want to get into a debate on 'breaking' the back button with full javascript/AJAX based sites, instead of having pages use regular navigation and only using AJAX within the same page.  Let's just assume you are here because you need to fix the back button for your Rails site after using link_to_remote throughout.  There are a number of solutions out there for various javascript toolkits, but I never was able to find anything that was terribly helpful for Prototype/Scriptaculous.  So I took some of the existing ones out there and adapted them to the Rails environment.

This code uses the same basic methodology as the others out there: manipulating the location bar URL so that it has a hash (#) in it with additional information about the AJAX link which creates back and forward history entries when it is changed. It uses an iframe to support this functionality in IE6 and IE7, otherwise using location.hash manipulation in Firefox, Opera, and Safari 3. A thread (PeriodicalExecuter) is created that checks for changes to the location bar and makes a new Ajax request if needed). Both single step back and forward buttons work, but using the back and forward drop downs will not. Reload/Refresh also works properly, as does Open in New Window/Tab right click and Bookmarks. It simply does normal link_to_remote processing for other browsers, such as old versions of Safari/WebKit (so no history support but the site still works).

The key difference between this method and the others however, is that instead of using some obscure mapping or number for the hash, we simply append all of the Rails controller/action/id?params information after the hash, so if you click an AJAX link http://my.domain.com/controller/action/id?param1=something your location bar will change to http://my.domain.com/#/controller/action/id?param1=something

///////////////////////////////////////////////////////////
// Slain Jamison Wilde
// This source code is released into the public domain


var PrototypeBackButton = {
    start: function(root, title) {
        this.browser_id = 0;
        this.history_clicks = 0;
        this.ajax_object = null;
        this.history_current_cache = "";
        this.root = root; // this is the initial page we are loading.
        this.title = title;

        is_backable_webkit = false;
        if (Prototype.Browser.WebKit) {
            navigator.userAgent.match(/AppleWebKit\/([^ ]+)/)
            if(parseInt(RegExp.$1) > 420) {
                is_backable_webkit = true;
            }    
        }

        if (Prototype.Browser.IE) {
            this.browser_id = 1;
            var history = $("ie_hash_history");
            var iframe = history.contentWindow.document;
            iframe.open();
            iframe.close();
            iframe.location.hash = location.hash + location.search; // ie breaks up the hash
           this.is_started = true;
           if (location.hash + location.search == "" || location.hash + location.search == "#") {
             new Ajax.Request(root, {asynchronous:true, history:true, evalScripts:true}); 
           }
           this.checker_thread = new PeriodicalExecuter(function() {
             PrototypeBackButton.url_hash_check();
           }, 0.2);  
        } else if (Prototype.Browser.Gecko || Prototype.Browser.Opera || is_backable_webkit == true ) {
          this.browser_id = 2;
          this.is_started = true;
          if (location.hash == "") {      
            new Ajax.Request(root, {asynchronous:true, history:true, evalScripts:true}); 
          }
          this.checker_thread = new PeriodicalExecuter(function() {
              PrototypeBackButton.url_hash_check();
           }, 0.2);  
        } else {
          this.is_started = false;
          new Ajax.Request(root, {asynchronous:true, evalScripts:true}); 
        }
    },

    // Destructor
     destroy: function() {
        this.stop();
    },


     set_root: function(root) {
        this.root = root;
    },

     set_title: function(title) {
        this.title = title;
    },

    stop: function() {
        if (this.checker_thread) { 
            this.checker_thread.stop();
            this.checker_thread = null;
      this.is_started = false;
        }
    },

     url_hash_check: function() {
        if(this.browser_id > 0 && document.title != this.title)
            // hack
            document.title = this.title;
        if (this.browser_id == 1) {
            var history = $("ie_hash_history");
            var iframe = history.contentDocument || history.contentWindow.document;
            var current_hash = iframe.location.hash + iframe.location.search; // ie splits it
            if(current_hash != this.history_current_cache || this.history_clicks > 0) {
        this.history_clicks = 0;
                location.hash = current_hash;
                this.history_current_cache = current_hash;
                if (this.ajax_object) {
                    this.ajax_object.request(current_hash.replace(/^#/, ''));
                    this.ajax_object = null; // dont reuse the request object
                } else {
                    new Ajax.Request(current_hash.replace(/^#/, '')); // reloads dont have an existing object
                }
            }
        } else if (this.browser_id == 2) {
            var current_hash = location.hash;
            if(current_hash != this.history_current_cache || this.history_clicks > 0) {
                this.history_clicks = 0;
                if (this.ajax_object) {
                    this.ajax_object.request(current_hash.replace(/^#/, ''));
                    this.ajax_object = null; // dont reuse the request object
                } else {
                    new Ajax.Request(current_hash.replace(/^#/, ''));  // reloads dont have an existing object
                }
                this.history_current_cache = current_hash;
            }
        }
    },

    ie_history_click: function(hash) {
        var newhash = '#' + hash;
        location.hash = newhash;
        var history = $("ie_hash_history");
        var iframe = history.contentWindow.document;
        iframe.open();
        iframe.close();
        iframe.location.hash = newhash;
    }

}
////////////////////////////////////////////////////////////

  request: function(url) {
        if (PrototypeBackButton.is_started && PrototypeBackButton.browser_id == 1 && this.options.history == true) {
            PrototypeBackButton.ajax_object = this;
            this.options.history = null; // prevent inf loop
            PrototypeBackButton.ie_history_click(url);
            PrototypeBackButton.history_clicks = 1; 
        } else if (PrototypeBackButton.is_started && PrototypeBackButton.browser_id == 2 && this.options.history == true) {
            PrototypeBackButton.ajax_object = this;
            this.options.history = null; // prevent inf loop
            parent.location = '#' + url;
      location.hash = '#' + url
            PrototypeBackButton.history_clicks = 1;
        } else {    
            this.url = url;
            this.method = this.options.method;
            var params = Object.clone(this.options.parameters);

            if (!['get', 'post'].include(this.method)) {
                // simulate other verbs over post
                params['_method'] = this.method;
                this.method = 'post';
            }

            this.parameters = params;

            if (params = Hash.toQueryString(params)) {
                // when GET, append parameters to URL
                if (this.method == 'get')
                    this.url += (this.url.include('?') ? '&' : '?') + params;
                else if (/Konqueror|Safari|KHTML/.test(navigator.userAgent))
                    params += '&_=';
            }

            try {
                if (this.options.onCreate) this.options.onCreate(this.transport);
                Ajax.Responders.dispatch('onCreate', this, this.transport);

                this.transport.open(this.method.toUpperCase(), this.url,
                    this.options.asynchronous);

                if (this.options.asynchronous)
                    setTimeout(function() { this.respondToReadyState(1) }.bind(this), 10);

                this.transport.onreadystatechange = this.onStateChange.bind(this);
                this.setRequestHeaders();

                this.body = this.method == 'post' ? (this.options.postBody || params) : null;
                this.transport.send(this.body);

                /* Force Firefox to handle ready state 4 for synchronous requests */
                if (!this.options.asynchronous && this.transport.overrideMimeType)
                    this.onStateChange();

            }
            catch (e) {
                this.dispatchException(e);
            }
        }
  },

def history_remote(name, options = {}, html_options = {})
    if $config['enable_back_button'] && $config['enable_back_button'] == true
        html_options[:href] = "#" + url_for(options[:url]) # lets 'open in new tab' etc work
        options[:history] = true
        function = remote_function(options);
        function.gsub!(/asynchronous\:true/, 'asynchronous:true, history:true')
#                 logger.dbg function
        link_to_function(name, function, html_options)
    else
        link_to_remote(name, options, html_options)
    end
end

<iframe id="ie_hash_history" style="display: none;"></iframe>