Saturday, 25 August 2012

Upload very large files - Part 4

Earlier in this blog I talked about the general idea behind one solution for uploading very large files (I consider very large when they are above 2GB), how the file API is becoming more popular and the possible client side approaches. This time I’ll describe the Silverlight and JavaScript communication for the client implementation part of the solution. As a quick reminder, the client is responsible for:

  • Prompt the file dialog and store locally the metadata related to the file to be uploaded.
  • Send each packet accordingly with its id, the current packet number and the total packets.
  • Receive the ACK from each packet and send the last received + 1 unless this is the last one.

With the pure JavaScript experience in the client implementation, I proceeded to make a clone using Silverlight technology as a fallback if the browser doesn’t have all the requirements for running the HTML5 implementation. I considered Flash but as I mentioned in my earlier post in this blog, I hate Flash, in addition, Silverlight became more popular nowadays and the project where I was working included it. Anyway, if someone wants to make a flash version, it will be welcome!

The visual element are pretty simple (and rough) just a Textbox and a Button surrounded by a Grid layout, the goal is just to simulate the input file control, when the user clicks on the button, a file dialog is open in the same way the browser does it.

With this approach, all the code that manages the sending and receiving data from the server was translated from JavaScript to C#; in order to simplify the code, I’ve done a class named AsyncWebRequest. Some interesting points with Silverlight implementation is the communication with JavaScript, the usual way to do this is by annotating the public properties and methods with the attribute [ScriptableMember] and the class with [ScriptableType], so the client can interact through this interface.

From the client side, first, I wanted to avoid carrying with the file Silverlight.js, so I had to inspect very deep what exactly that file does, well, really not much: some helper functions that write the object tag with all the params and something interesting that of course I copy for me, the events setup. As far as I was made research, there’s a param named onLoad where we assign the name of the load function.

That works well if we have only one Silverlight in the page or if we have the complete control over what is rendered in the page. My idea was that you can have as many freshuploads as you want (you will know why). A little trick must be done, instead of expecting that a function exists, I added to window object (in order to be visible to the Silverlight object) but I added to the function name a random auto incremented number, so every newly freshupload created, will add a completely different name to the function load, and this function is implemented internally, and through this function as parameter we have the host object and with it, all the public methods and properties.

Once this key element is established, the code is barely a wrapper of the internal Silverlight implementation, and it looks as follows:

prototype: {
    init: function () {
        // ...
    },
    getDOMElement: function () {
        return this.input.next();
    },
    hasFile: function () {
        return this.uploader.HasFile();
    },
    getFileName: function () {
        return this.uploader.GetFileName();
    },
    getFileSize: function () {
        return this.uploader.GetFileSize();
    },
    cancelUpload: function () {
        this.uploader.CancelUpload();
    },
    startUpload: function () {
        this.uploader.StartUpload();
    }
}
	

In the next post I’ll describe how this can be used seamlessly, without worrying about what implementation to use, I mean, some kind of “Factory” that determines which variant to use. I’ll also show here in further posts some detailed examples on how to use it either with ASP.NET MVC or ASP.NET WebForms.

The whole source code can be found at https://bitbucket.org/abelperezok/freshupload.

Upload very large files - Part 3

Earlier in this blog I talked about the general idea behind one solution for uploading very large files (I consider very large when they are above 2GB), how the file API is becoming more popular and the possible client side approaches. This time I’ll describe the JavaScript client implementation part of the solution. As a quick reminder, the client is responsible for:

  • Prompt the file dialog and store locally the metadata related to the file to be uploaded.
  • Send each packet accordingly with its id, the current packet number and the total packets.
  • Receive the ACK from each packet and send the last received + 1 unless this is the last one.

The initial implementation, some kind of proof of concept, was made using HTML5 with the FileReader API, but as this cannot be the only implementation, it was necessary to let open an “interface” so any other technology for further implementations can be done without the risk of touching unnecessary code fragments.

The object was named $.uploaderHtml5 as a jquery extension and the interface was defined as follows:

{    
    // initialization code
    init: function () { },
    
    // return the visual element associated
    getDOMElement: function () { },
    
    // return true if there's a selected file
    hasFile: function () { },
    
    // return the file name from the input dialog
    getFileName: function () { },
    
    // return the file size from the input dialog        
    getFileSize: function () { },
    
    // stop the current upload 
    cancelUpload: function () { },
    
    // start the current upload
    startUpload: function () { }
}
 

In the HTML5 implementation there are of course more internal functions that deal with the proper send and receive packets actions, because it’s responsible for all the verifications such as parse the ACK from the server and take action accordingly, the POST to the server must be done with all the data required so the server can interpret it and ask for a new piece.

The particularly interesting point here is how the file API is used to deal with the partial and vendor-dependent implementation, the “standard” method slice has three known variants: mozSlice, webkitSlice and slice, so there’s no choice that ask for each of them.

if ('mozSlice' in this.ufile) {
    // mozilla
    packet = this.ufile.mozSlice(startByte, endByte);
} else if ('webkitSlice' in this.ufile) {
    // webkit
    packet = this.ufile.webkitSlice(startByte, endByte);
} else {
    // IE 10
    packet = this.ufile.slice(startByte, endByte);
} return packet;
 

Another HTML5 element used in this code is the FormData object which is really easy to use for send post data to the server; in addition, the XMLHttpRequest has the ability of uploading files which is very important in order to simplify the code, otherwise an iframe should have been set up with a form inside with an input type=file.

The whole source code can be found at https://bitbucket.org/abelperezok/freshupload.