A bug hidden for 4 years !

I have to admit I've abandoned my file uploader for some time, and I've been revisiting the code recently as part of some experiments with .NET Core 2.

After some work done to make it work with the new framework, to my surprise, it just looked to work perfectly until I tried to validate one of the basic implicit requirements: the uploaded copy must be exactly the same as the original file. It turned out that even if the files had the same exact size, to the byte, they weren't identical, how was that possible ?

Symptom:

After uploading a file larger than the packet size (default to 4 MB), the resulting copy was not identical to the original.

Simple test:

Upload a zip file larger than 4 MB, try to open it. It was corrupt.

The most uprising fact was that if I ran the .NET 4 version it still worked as expected ! Next, I migrated the solution to .NET 4.7 and still worked as expected, apparently it was only affecting to .NET Core version. So, under the circumstance, I tried to compare request / response side by side working version vs bugged version.

A few things looked a bit suspicious:

• some extra headers sent on .NET working version
• Transfer-Encoding: chunked from .NET Core bugged version

None of those were related. Back to square one.

I tried reducing the packet size to 1024 bytes and upload a text file so I could see how the content differed but no luck, it was indeed the same (verified comparing documents with Diff tool) Well now deep debugging showed that the buffer wasn't being written completely to output file, that would explain the corrupted resulting file but not the size difference. Let's see the code.

Original version

var buffer = new byte[contentLength];
var bytesread = inputStream.Read(buffer, 0, buffer.Length);
if (bytesread > 0)
{
    using (var fs = File.Open(actualfile, FileMode.Append, FileAccess.Write))
    {
        fs.Write(buffer, 0, buffer.Length);
    }
    //json.Data = new {guid, packet};
    return new UploadPacketResult(guid, packet);
}

Spot something weird ?

It turns out that Stream.Read can perform a blocking operation that reads up to the length specified but not necessarily the whole buffer, now it's obvious! but how it didn't fail before !! I don't have an answer for that :)

Well, here is the fixed version

var buffer = new byte[contentLength];
using (var fs = File.Open(actualfile, FileMode.Append, FileAccess.Write))
{
    int read = 0;
    while ((read = inputStream.Read(buffer, 0, buffer.Length)) > 0)
    {
        fs.Write(buffer, 0, read);
    }
}
//json.Data = new {guid, packet};
return new UploadPacketResult(guid, packet);

And that's the way it was supposed to be done in the first place, but there was no test case that proved otherwise (until now, 4 years later)

Upload very large files - Part 4

Earlier in this blog I talked about the general idea behind one solution for uploading very large files (I consider very large when they are above 2GB), how the file API is becoming more popular and the possible client side approaches. This time I’ll describe the Silverlight and JavaScript communication for the client implementation part of the solution. As a quick reminder, the client is responsible for:

• Prompt the file dialog and store locally the metadata related to the file to be uploaded.
• Send each packet accordingly with its id, the current packet number and the total packets.
• Receive the ACK from each packet and send the last received + 1 unless this is the last one.

With the pure JavaScript experience in the client implementation, I proceeded to make a clone using Silverlight technology as a fallback if the browser doesn’t have all the requirements for running the HTML5 implementation. I considered Flash but as I mentioned in my earlier post in this blog, I hate Flash, in addition, Silverlight became more popular nowadays and the project where I was working included it. Anyway, if someone wants to make a flash version, it will be welcome!

The visual element are pretty simple (and rough) just a Textbox and a Button surrounded by a Grid layout, the goal is just to simulate the input file control, when the user clicks on the button, a file dialog is open in the same way the browser does it.

With this approach, all the code that manages the sending and receiving data from the server was translated from JavaScript to C#; in order to simplify the code, I’ve done a class named AsyncWebRequest. Some interesting points with Silverlight implementation is the communication with JavaScript, the usual way to do this is by annotating the public properties and methods with the attribute [ScriptableMember] and the class with [ScriptableType], so the client can interact through this interface.

From the client side, first, I wanted to avoid carrying with the file Silverlight.js, so I had to inspect very deep what exactly that file does, well, really not much: some helper functions that write the object tag with all the params and something interesting that of course I copy for me, the events setup. As far as I was made research, there’s a param named onLoad where we assign the name of the load function.

That works well if we have only one Silverlight in the page or if we have the complete control over what is rendered in the page. My idea was that you can have as many freshuploads as you want (you will know why). A little trick must be done, instead of expecting that a function exists, I added to window object (in order to be visible to the Silverlight object) but I added to the function name a random auto incremented number, so every newly freshupload created, will add a completely different name to the function load, and this function is implemented internally, and through this function as parameter we have the host object and with it, all the public methods and properties.

Once this key element is established, the code is barely a wrapper of the internal Silverlight implementation, and it looks as follows:

prototype: {
    init: function () {
        // ...
    },
    getDOMElement: function () {
        return this.input.next();
    },
    hasFile: function () {
        return this.uploader.HasFile();
    },
    getFileName: function () {
        return this.uploader.GetFileName();
    },
    getFileSize: function () {
        return this.uploader.GetFileSize();
    },
    cancelUpload: function () {
        this.uploader.CancelUpload();
    },
    startUpload: function () {
        this.uploader.StartUpload();
    }
}
	

In the next post I’ll describe how this can be used seamlessly, without worrying about what implementation to use, I mean, some kind of “Factory” that determines which variant to use. I’ll also show here in further posts some detailed examples on how to use it either with ASP.NET MVC or ASP.NET WebForms.

The whole source code can be found at https://bitbucket.org/abelperezok/freshupload.

Upload very large files - Part 3

Earlier in this blog I talked about the general idea behind one solution for uploading very large files (I consider very large when they are above 2GB), how the file API is becoming more popular and the possible client side approaches. This time I’ll describe the JavaScript client implementation part of the solution. As a quick reminder, the client is responsible for:

• Prompt the file dialog and store locally the metadata related to the file to be uploaded.
• Send each packet accordingly with its id, the current packet number and the total packets.
• Receive the ACK from each packet and send the last received + 1 unless this is the last one.

The initial implementation, some kind of proof of concept, was made using HTML5 with the FileReader API, but as this cannot be the only implementation, it was necessary to let open an “interface” so any other technology for further implementations can be done without the risk of touching unnecessary code fragments.

The object was named $.uploaderHtml5 as a jquery extension and the interface was defined as follows:

{    
    // initialization code
    init: function () { },
    
    // return the visual element associated
    getDOMElement: function () { },
    
    // return true if there's a selected file
    hasFile: function () { },
    
    // return the file name from the input dialog
    getFileName: function () { },
    
    // return the file size from the input dialog        
    getFileSize: function () { },
    
    // stop the current upload 
    cancelUpload: function () { },
    
    // start the current upload
    startUpload: function () { }
}
 

In the HTML5 implementation there are of course more internal functions that deal with the proper send and receive packets actions, because it’s responsible for all the verifications such as parse the ACK from the server and take action accordingly, the POST to the server must be done with all the data required so the server can interpret it and ask for a new piece.

The particularly interesting point here is how the file API is used to deal with the partial and vendor-dependent implementation, the “standard” method slice has three known variants: mozSlice, webkitSlice and slice, so there’s no choice that ask for each of them.

if ('mozSlice' in this.ufile) {
    // mozilla
    packet = this.ufile.mozSlice(startByte, endByte);
} else if ('webkitSlice' in this.ufile) {
    // webkit
    packet = this.ufile.webkitSlice(startByte, endByte);
} else {
    // IE 10
    packet = this.ufile.slice(startByte, endByte);
} return packet;
 

Another HTML5 element used in this code is the FormData object which is really easy to use for send post data to the server; in addition, the XMLHttpRequest has the ability of uploading files which is very important in order to simplify the code, otherwise an iframe should have been set up with a form inside with an input type=file.

The whole source code can be found at https://bitbucket.org/abelperezok/freshupload.

Upload very large files - Part 2

Earlier in this blog I talked about the general idea behind one solution for uploading very large files (I consider very large when they are above 2GB), how the file API is becoming more popular and the possible client side approaches. This time I’ll describe the server side part of the solution. As a quick reminder, the server is responsible for:

• To determine if the request is to be handled by the mechanism, if not, just let it go.
• To determine if the request is an initial packet or a data packet.
• To handle properly each type of packet giving the appropriate response so the client can continue.

This is part of a bit more complex solution, named FreshUplaod, which has the Server component and client components using either JavaScript or Silverlight. The core element is the UploadHelper class, as the name indicates is a helper, it encapsulates the dirty logic regarding the request data reading and processing all stuff.

The methods ProcessInitialPacket and ProcessDataPacket do the actual work, but it’s necessary to know if the request data corresponds to one or another, that’s the job for these other methods IsInitialPacket and IsDataPacket. The actual helper utility is another method, ProcessRequest that acts as a façade to the whole mechanism. An example on how to use it is as follows:

[HttpPost]
public ActionResult Upload(FormCollection collection)
{
    var packetSize = 4 * 1024 * 1024; // default to 4 MB
    var filePath = Server.MapPath("~/_temp_upload/");

    var result = UploadHelper.ProcessRequest(
        Request, filePath, packetSize);

    if (result != null)
        return Json(result);
    return Content("");
}
 

This is using a controller action on ASP.NET MVC 3, where the access to the request is quite easy and the conversion to Json is stress-free too. The key points here are the packet size and the folder where the file is to be stored, the packet size must be exactly the same configured on client side in order to expect a good behavior, I’ve used the 4 MB value because this is the default constraint imposed by ASP.NET and of course this can be tweaked as you want in web.config file.

Now let’s see how to use this element on ASP.NET WebForms, the idea I suggest here is to use a handler whether it was a Generic Handler or ASP.NET Handler, it can be also a WebForm but is not elegant enough solution because the WebForms are chained in a different pipeline and there can be a bit of noise, although it work as well. Here’s the code sample:

public void ProcessRequest(HttpContext context)
{
    var packetSize = 4 * 1024 * 1024; // default to 4 MB
    var filePath = context.Server.MapPath("~/_temp_upload/");

    var result = UploadHelper.ProcessRequest(
        context.Request.RequestContext.HttpContext.Request, 
        filePath, packetSize);

    if (result != null)
    {
        JsonResult(context.Response, result);
    }
    else
    {
        context.Response.Write("");
    }
}

private void JsonResult(HttpResponse response, object result)
{
    response.ContentType = "application/json";
    var serializer = new JavaScriptSerializer();
    var json = serializer.Serialize(result);
    response.Write(json);
}
 
 

Here we have a little more hand work but not so much, the json conversion and output is done completely manual (if some WebForms people knows how to achieve this with fewer code, please let me know cause I’m a little out of this part of ASP.NET, although I support Scott Hanselman with his slogan that “Only One ASP.NET”). The other key point is how to get the Request but with an instance of HttpRequestBase class, the default context.Response is an HttpRequest instance, so after some minutes of try-error-retry I found this way to get right value, again if some knows a better way to do it, you know.

In the next posts I’ll cover the other parts of this solution. The whole source code can be found at https://bitbucket.org/abelperezok/freshupload, I am still preparing the code samples so in a few weeks they will be ready.

Upload very large files - Part 1

In my last post I talked about settings we have to set server side on IIS in order to allow large uploads, and I also mentioned the security risks involved. The solution I’ll talk this time is becoming more common as HTML5 is being implemented more and more.

The main problem was related to the size of the post when uploading, but why not split file into several small pieces and upload piece by piece? Maybe some time ago was crazy to think about manipulating the file in client side from JavaScript, the only ways were using Flash or Silverlight, now the file API is more popular, almost all modern browsers implement it but IE which is always retarded implementing good stuff.

I’ve found some implementations such as plupload or http://hertzen.com/experiments/jsupload/ but I’m not 100 % satisfied with it, I took the ideas and hands on. In this first part I’ll explain how it works to achieve a bullet proof mechanism, for example: The connection might be interrupted or something can happen on the server and the worker thread was shot down. The mechanism must be protected against these events.

I talked to a friend of mine and after a session of brainstorming I decided to imitate some kind of TCP connection in regard to sending packet and receiving the ACK (Acknowledgement), this stuff. The idea is implemented as follows:

• The browser opens the file dialog and a file is selected, the initial packet is prepared with file name, the type (MIME) and the size.
• In the server a new GUID is generated for this file that will be the file id for the whole process.
• A new file is created with this id that will be the metadata file, where the information from the client will be stored for further use.
• A new blank file is created with this id and the extension of the original name that will be the data file.
• The reply for this initial packet is a JSON with the GUID generated.

The following steps in client side are:

• Read the file fragment according to the current packet number and send to the server the data packet with the id, the current packet number and the total packets.
• Each data packet sent receives its ACK which indicates the number of the last packet received OK, after that the client will send the last received + 1 unless this is the last one.

The server process the data packet:

• If the metadata file or actual file doesn’t exist, the result is an ACK with packet -1, which means the file must be uploaded from scratch.
• If the file doesn’t have the required size: the packet number – 1 multiplied by packet size, then return the required value in order to the client knows which is the correct fragment to resend.
• If everything goes well, a success ACK with the current packet number is sent encouraging to the client to send the next packet.

I’ve implemented this using HTML5 techniques but not all the browsers are capable for to do it, Modernizr could be a great helper on this, since there’s no HTML5 detection but feature detection on demand, for now it’s necessary the browser to be capable of use the file API and if you want to support retry after refresh the page, it should be capable of access to local Storage which allows to store data and retrieve it when necessary. The browser will able to use the XMLHttpRequest object for upload files.

I haven’t implemented yet any fallback mechanism, it will be nice to have a third party such as Silverlight which could substitute the missing browser HTML5 features, a good example of implementation using Silverlight can be found at HSS.Interlink. they cover all the aspects on read a fragment of file and send the request to the server.

My implementation I am planning to distribute it via nuget.org containing a js script and a dll with the classes to be called from any handler/controller without to have to deal with all the verifications mentioned above.

In next posts I’ll cover the implementation of this solution and I’ll provide a code sample for to understand it better.

Magic settings for upload large files under IIS and .NET

I remember when I started this blog; I talked about upload large files using .NET and IIS. This time I’ll share with you some interesting points to be aware of when trying to upload large files. I consider large when the size is bigger than 2GB, because this is the actual limit is almost all web servers under Microsoft technology.
Based on my experience with this kind of situations, I can say that the web environment is not ready to transfer that much information. Remember the beginnings of WWW, when everything that roamed through the wires were just text and small graphics, I say more, remember what HTTP stands for (Hypertext Transfer Protocol) and what is HyperText?
However, the file transfer to and from the server is something very common nowadays, therefore, it’s necessary to know how to handle it. For those who doesn’t need to upload files greater than 500 MB, I recommend the solution I presented through the my firsts posts in this blog, even for upload a bit bigger files, but when the size start over the 1 GB it’s better to think about another solution.
The larger sizes you could upload the bigger security risk you have. I explain, solutions like NeatUpload and many others (open source or not) expect to make the upload in a big single request, although there are modules that make the copy block by block in order to be sure of not making big memory peaks, the web server must allow a very large request, exposing it to a DOS attack, just an example. Another possible problem may be if the client doesn’t have a fast connection (like me) there’s a (high) risk that a large file might be interrupted, and then? Start over again. On the server side the risk of execution timed out exists too (I’ve faced that), so the value of this setting must be increased.
If you wish to upload small/medium files there’s no problem using this kind of solution, but if you want go over the size, I suggest another kind of workaround.
Alongside my research for the best solution I’ve found the following links I hope to be useful to you, I let MSDN articles, Stack Overflow questions, among others very interesting sources written by the most involved people in this matter, where one can drink the honeys of knowledge.

• http://blogs.msdn.com/b/prashant_upadhyay/archive/2011/07/13/large-file-upload-issue-in-asp-net.aspx
• http://www.iis.net/ConfigReference/system.webServer/security/requestFiltering/requestLimits
• http://www.webtrenches.com/post.cfm/iis7-file-upload-size-limits
• http://stackoverflow.com/questions/6802519/how-to-upload-files-over-2gb-to-iis-7-5-net-4
• http://weblogs.asp.net/jgalloway/archive/2008/01/08/large-file-uploads-in-asp-net.aspx
• http://www.mojoportal.com/supporting-large-file-uploads.aspx

As a quick conclusion, I can say the following in order to configure properly your application:
The default value for the request size in ASP.NET 2.0 and later is 4 MB. Under IIS6 the setting is configured by the httpRuntime node, example:

 <configuration> 
  <system.web>
   <httpRuntime maxRequestLength="102400" executionTimeout="3600" />
  </system.web>
 </configuration> 

Important to say here, the value of maxRequestLength is given in KB and the executionTimeout is given in seconds.
Under IIS7+ the setting is configured by the requestLimits node, example:

 <system.webServer>
  <security>
   <requestFiltering>
    <requestLimits maxAllowedContentLength="1024000000" />
   </requestFiltering>
  </security>
 </system.webServer>

In this case the value of maxAllowedContentLength is given in bytes. This must combined with the applicationHost.config file usually located at (C:\Windows\System32\inetsrv\Configs) whose section should look like:

 <section name="requestFiltering" overrideModeDefault="Allow" />

As I said before, I hope this compilation saves you time of google-research, in a further post I’ll talk about another solution that I consider that more fits better in web environment.

jQuery.NeatUpload – Examples refactored

jQuery.NeatUpload – Release 0.4

I’ve been using the jquery plugin for neatupload and the bugs keep coming up, well, that’s the developer’s life. Catching bugs is an ability that every developer must have and more than that, do not shame on, but be proud of.
This time I’ve improved the error handling when the server send as response some no comprehensive, for instance, there is a limit set in web.config for the maximum file size to upload and when the client attempts to send a file bigger than the limit, the server replies with some bizarre response.
As part of the mechanism implemented, it expects the HTML response for parse it and determine what to do, I mean, send information to the client indicating whether the request failed or not. Sometimes the HTML sent by the server was impossible to read because of some mysterious JavaScript error “Access denied”. I had no time for digging deeper in this matter, so I try-catch it and just a little extra setup in order to be consequent and ready. The release 0.4 can be downloaded here.

jQuery.NeatUpload – Examples refactored

In order to prepare for launch a new release of NeatUpload including these JavaScript features I’ve been doing some refactoring in the samples I published a few months ago. The first thing I’ll do is to apologize with the WebForms people, because I’ve let them in the road with the examples, just the part 1. Now the updated versions have both implementations WebForms and MVC3. Of course these new versions also have applied the bugfix mentioned above.
There’s something new in the web.config files, this time they are prepared for either IIS6 or IIS7+ and highly commented (well, this is questionable and relative to anyone’s need) every setting needed for set up NeatUpload even clarifying the confusing and inconsistent size units that Microsoft has implemented in both versions of IIS, they are in bytes and Kilobytes respectively.
Before describe what’s in these examples, I want to alert you about something that I’ve been dealing with many times and that has been cause of mysterious error in production scenarios. There is a setting in web.config that indicates the maximum time that IIS can wait before it shuts down the thread and return time out error. This is under and it’s expressed in seconds, this is especially useful when the files to be uploaded are really big.

What’s in the examples?

• Default values: Shows how to use the minimum setup code and use the default values.
• Changed Add and Upload texts: Shows how to change the texts for the elements that add and/or upload items rather than the default internal texts.
• Changed Add and Upload elements: Shows how to change the default link for add and/or upload element by custom elements like buttons.
• Custom cancel elements. Part 1: Shows how to change the cancel element by creating a simple new element inline and binding it to the internal click event.
• Custom cancel elements. Part 2: Shows how to change the cancel element by cloning a complex element and binding it to the internal click event.
• Start Upload Automatically: Show how to start automatically the upload without using a queue and adding one by one. This example uses just add element.
• Show Progress Bar: Shows how to use a more cute visual feedback by styling up a progress bar that indicates the progress and changes the color if an error occurs.
• Validating Data: Shows how to intercept the action of send the file to the server by performing validations to the associated data.
• Sending extra Data: Shows how to send associated data along with the file by serializing it as JSON and how to receive it properly on the server.

à la prochaine mes amis!

NeatUpload jQuery plugin – Examples – Part 3

In my last post I covered three scenarios where we changed the cancel the link element and start upload immediately. Well, let's continue tweaking, in this case the turn is for the user experience, I’ll show how to display a fancy progress bar while the upload operation is in progress, validate elements before send the file and even send extra data attached to the uploaded file. Let’s start.

Scenario 5 – Showing a fancy progress bar.

In all previous examples I have shown a simple number followed by a “%” symbol to indicate the progress, but I real life the user experience should be more than that, so all we are accustomed to is to see a progress bar increasing its size horizontally until the maximum width is reached, or tell me if I am crazy!? At least Graphical User Interfaces have used this pattern for years and it works. So if we intercept the moment when the file is added to the queue, it’s possible take the control and to construct everything necessary for display the progress. Let’s check the code:

<link href="@Url.Content("~/Content/progress.css")" rel="stylesheet" type="text/css" />

fnShowStatus: function (plugin, index, data) {

    $('table[id=tablequeue] > tbody > tr[data-neatupload-rowprog=' + index + '] div.progress-bar').width(data.PercentComplete + "%");

},

fnAddFile: function (plugin, filename, form, index) {

    var cancel = plugin.options.createCancelLink(plugin, index);

    var row1 = $("<tr data-neatupload-rowname='" + index + "' class='uploadItem-uploading'></tr>");

    var row2 = $("<tr data-neatupload-rowprog='" + index + "' class='uploadItem-uploading'></tr>");

    $('table[id=tablequeue] > tbody')

        .append(row1).append(row2);

    row1.append($("<td style='width: 80%'></td>")

        .append("<div class='uploading-video-name'>" + name + "</div>")

        .append("<div class='uploading-video-file-name'>" + filename + "</div>"))

    .append($("<td></td>"));

    row2.append($("<td></td>")

        .append($("<div class='barcontainer'></div>")

        .append("<div style='width: 0%' class='progress-bar'>")))

    .append($("<td class='cancel'></td>").append(cancel));

},

fnRemoveFile: function (plugin, index, reason, html) {

    var row1 = $('table[id=tablequeue] > tbody > tr[data-neatupload-rowname=' + index + ']');

    var row2 = $('table[id=tablequeue] > tbody > tr[data-neatupload-rowprog=' + index + ']');

    if (reason == "canceled") {

        row1.removeClass("uploadItem-uploading").addClass("uploadItem-canceled");

        row2.removeClass("uploadItem-uploading").addClass("uploadItem-canceled");

        $('div.progress-bar', row2).width("100%");

    }

    $("td.cancel", row2).html("<span>" + reason + "</span>");

}

<div>

    <table width="100%" class="uploadItem" id="tablequeue">

        <tbody>

        </tbody>

    </table>

</div>

The first this I remark here is the css inclusion, of course for achieve nice things it’s necessary a good inspiration, which I don’t have, I confess it, but something not so ugly it’s done with the css provided in the example, as usually you’re free to play with it.

Here is the same function we’ve overridden in last scenario, fnAddFile, which is executed right after the user clicks the button Add. The main idea with that amount of code is create tow table row for each file in the queue, the first for show the file name and the second the progress and the cancel element, previously discussed. Please note, we have used the internal function to create the cancel element whatever to be its implementation (overridden or default).

The function fnShowStatus this time with a little jquery magic finds the progress element and updates its width instead of write directly the percent value. Finally the function fnRemoveFile ad you can guess is responsible of remove the rows involved with the file to be removed, as an apart the parameters for this function: plugin is a reference to the plugin as usually, index is the identifier related to the current file in the queue, reason is a string indicating the reason of removing the current file (canceled or completed) and the html parameter give us the data returned by the server after finished the work. In this case we have checked if the reason is canceled then fill the width and a different color in the progress bar for tell to the user that that upload did not success.

At the end of the example we find a div with a table inside and this table has an id = tablequeue and an empty tbody, this is the placeholder for the rows created in runtime by the code previously explained.

Scenario 6 – Validating data before to perform the upload.

Sometimes we have a form with some data and the upload can be performed only if the form is consistent with some rules and even attach some data, but I let this for the next scenario. For verify these rules I understand (and I think you too) to validate the input fields. Surprisingly I’ve anticipated this kind of behavior and there is a function that may be overridden whose name is fnValidate and receives a plugin object reference as parameter. Inside this function body you are able to use any validation engine such as jquery.validate or whatever you want, for didactic purposes I just made a very simple validation.

(...)

fnValidate: function (plugin) {

    var valid = $("#name").val() && $("#age").val();

    if (!valid) alert("Incorrect Data!");

    return valid;

}

(...)

<div>

    <label for="name">Name</label>

    <input id="name" name="name" type="text" value="" />

</div>

<div>

    <label for="age">Age</label>

    <input id="age" name="age" type="text" value="" />

</div>

As you can see, all I’ve done is test if the value of the name and age fields are not empty, in case of at least one of them is empty an alert box is shown. The rest of the plugin expects if you override this function to return true if everything is ok and false if there is something wrong with the input data.

Scenario 7 – Start upload immediately.

Sometimes we have to send some data to the server attached to the file, for example metadata associated to the file being uploaded, for solve this issue we can make a little trick in the fnAddFile function before send the data. We can gather the data and helping with the JSON object which is capable of serialize any JavaScript object into a string, and then that string is added to the form that actually send to file to the server. Remember that each file sent to the server is an actual POST with an actual form. In this case that form is received by parameter and it’s simply to create a new element, the ideal form field is the hidden field for this task and that’s what I do in the following code.

fnAddFile: function (plugin, filename, form, index) {

    var name = $("input[id=name]").val();

    var age = $("input[id=age]").val();

    $("input[id=name]").val("");

    $("input[id=age]").val("");

    var data = JSON.stringify({ name: name, age: age });

    $("<input type='hidden' name='filedata' value='" + data + "' />")

         .appendTo(form);

},

fnRemoveFile: function (plugin, index, reason, html) {

    alert(html);

}

@Html.Hidden("upload-action", Url.Action("UploadFileExtraData"))

The function fnRemoveFile that we used in the previous scenario, this time just show up with an alert box the html parameter, with the data returned by the server in the controller action that receives the post with the files. Please note here that I show the hidden field which saves that action, and note that is different than the previous scenarios, this time there’s something to do also in server side.

[HttpPost]

public ActionResult UploadFileExtraData(FormCollection collection)

{

    var tempPath = Server.MapPath("~/_temp_upload/");

    var value = string.Empty;

    for (var i = 0; i < UploadModule.Files.Count; i++)

    {

        value = UploadModule.Files[i].FileName;

        UploadModule.Files[i].SaveAs(tempPath + value);

        var data = collection["filedata"];

        var serializer = new JavaScriptSerializer();

        var deserializeObject = (Dictionary<string, object>)serializer.DeserializeObject(data);

        var name = (string)deserializeObject["name"];

        var age = (string)deserializeObject["age"];

        value = "Name = " + name + ", Age = " + age;

    }

    return Content(value);

}

I just show the controller action code (sorry webforms people, but don’t worry it’s the same in essence), this time there is a bit more of code in server side than previously. First thing new is to get from the collection, well it’s the same that take it from Request.Form[“…”], the important thing here is that we follow a convention the name filedata must match with the name we gave to the hidden field we created on the fly in fnAddFile and that’s why I marked in yellow background. The rest is simply deseralize it with the .NET powerful tools for that and finally store in the variable value the demonstrative string with the values extracted from the metadata and returned to the client.

Well, I think it’s all for the moment regarding the jquery.neatupload plugin. I hope this information to be useful for you. Any comment, feedback or even suggestion for improve it will be welcome. I’ll come back with some ASP.NET MVC 3 tricky features soon.

So Happy New Year to every one!

NeatUpload jQuery plugin – Examples – Part 2

In my last post I covered two scenarios where we changed the texts and the elements for the actions add and upload. Well, let's continue with these changes, in this case the turn is for the link of cancel operation. It's a bit more complex than the others, that's why I've let for another post, but no so hard to do.
Scenario 3 – Changing the cancel link.

Due to the fact the cancel link has a special function and is intended for call an internal function, it’s required that the custom cancel element to call sendCancel function on its click event. This function can be accessed through plugin parameter and options object. Let’s check the code:

@{

    // get the text from wherever i.e. some i18n service in the application
    var textAdd = "Add a file to upload list";
    var textUpload = "Start Upload";
    var textCancel = "Abort Operation";
}
<script src="@Url.Content("~/Scripts/jquery-1.5.1.min.js")" type="text/javascript"></script>
<script src="@Url.Content("~/Scripts/jquery.neatupload.js")" type="text/javascript"></script>
<script type="text/javascript" defer="defer">
    $(function() {
        $("#file1").neatupload({
            postDataUrl: $("input[type=hidden][id=upload-action]").val(),
            fnShowStatus: function (plugin, index, data) {
                $('span#progress-file').text(data.PercentComplete + "%");
            },
            textCancel: "@textCancel",
            selectorButtonAdd: "input[name=addfile]",
            selectorButtonUpload: "input[name=uploadfile]",
            createCancelLink: function (plugin, index) {
                var cancel = $('<div>' + plugin.options.textCancel + '</div>')
                        .click(function () {
                                                plugin.options.sendCancel(plugin, index);
                        });
                return cancel;
            }
        });
    });
</script>

@using(Html.BeginForm("UploadFile", "Home"))
{
    @Html.Hidden("upload-action", Url.Action("UploadFile"))
    <div>
        @Html.Label("newfield", "Select File: ")
        <div id="file1">
            place holder for the input -> file for upload
        </div>
        <input type="button" name="addfile" value="@textAdd" />
        <input type="button" name="uploadfile" value="@textUpload" />
    </div>
    <div>
        <span id="progress-file"></span>
    </div>
    <div>
        <input type="submit" value="save"/>
    </div>
}
Here is a new parameter introduced in the last release: createCancelLink, which corresponds to a function with 2 parameters: plugin and index.

·         plugin is a reference to the current plugin element and is used for access the functions with the current data.

·         index is the unique identifier for the current file being queued for upload.

In the example just created a “div” element with a custom text and set the click event to an anonymous function which calls to plugin.options.sendCancel(plugin, index); and that’s the key point, through plugin.options it’s possible access to functions declared as public in the plugin (as in any other jquery plugin). The current values of plugin and index are passed to sendCancel internal who is responsible for send a “cancel” message to the server and stops the internal frame (in case of upload is in progress).

Scenario 3.1 – Changing the cancel link using a more complex DOM element.
In the previous example I demonstrated how to change the element cancel, and that works will if this element is very simple (as in the example, of course), but what if we have an element previously created in the page? I suggest this code fragment for solve it.

(...)
createCancelLink: function (plugin, index) {
    var cancel = $("input[name=cancelupload]").clone();
    cancel.show().click(function () {
        plugin.options.sendCancel(plugin, index);
    });
    return cancel;
}
(...)
<input type="button" name="cancelupload" value="@textCancel" style="display: none" />

Here I have an input button named “cancelupload” hidden by default using inline style and this will be the element chosen for cancel. The first is get it via query (remember, we are using jquery so take all advantages) and make a clone, then show it and finally assign the click function for the event handler like the previous example.
Scenario 4 – Start upload immediately.
Sometimes we don’t want the user follow the two steps: add and upload, This is useful for applications where the user only upload one file at time, for example: attach a file in a webmail interface, set up a profile logo image, etc. I have a solution for that and I show you as follows:

fnAddFile: function (plugin, filename, form, index) {
    $(plugin.options.selectorButtonUpload).click();
}

I introduce you a new parameter fnAddFile, this is a function that is called when a new file is going to be added to the queue. By default this function renders a visual element where we can see the cancel element, the name of the file being queued and a placeholder for the progress element.

In this case I have overridden this default behavior and I just call the click event of upload button element using the plugin.options.selectorButtonUpload parameter previously explained, I know that could be called as a little dirty but I think is valid, if we already have defined a button and its click event with a lot of work done, why not call it and go? At least this is my concept of reuse of code.

Well, I think it’s all for today, in the next post I’ll talk the third group of scenarios for use this plugin, which include personalizing the user interface using a more cute progress bar and more use of the uploaded file in server side. I hope this information to be useful for you. The source code for this example is available here.

NeatUpload jQuery plugin – Examples – Part 1

Sorry for the delay but it’s been a busy month, well, in my previous post I promised some examples in different scenarios on how to use the plugin. So, let’s start with the simplest scenario:

Scenario 1 – Using default functionality and default visual appearance.
The basic structure is:
$(function() {
    $("#file1").neatupload({
        postDataUrl: "path/to/the/server/",
        fnShowStatus: function (plugin, index, data) {
            // handle the data progress values in order to make some visual feedback.
        }
    });
});

Here, we have the two essential parameters:

·         postDataUrl, the path for the elements in server side who is going to handle the POST.
·         fnShowStatus, in fact, this is not a required parameter but this function is ideal to keep visual feedback of uploading process, and here we have three parameters in the function:
o   The plugin object, available for call any function.
o   The file index, usually used for keep track on which file is, it is increased each time an upload in queued.
o   The data, JSON object with useful data for show on screen, such as: status, percent, speed, etc. Here is an example.
{

"Status":"Completed",

"BytesRead":806688,
"BytesTotal":806688,

"PercentComplete":100,

"BytesPerSec":12927672,

"Message":"",

"SecsRemaining":0,

"SecsElapsed":0,

"CurrentFileName":" picture01.jpg",

"ProcessingState":null,

"Files":

[

{

"ControlUniqueID":"fileField",

"FileName":"picture01.jpg",

"ContentType":"image/jpeg",

"ContentLength":806466

}

]

}

In the previous examples I’ve done something like this: 
$(function() {
        $("#file1").neatupload({
        postDataUrl: $("input[type=hidden][id=upload-action]").val(),
        fnShowStatus: function (plugin, index, data) {
            $('span#progress-file').text(data.PercentComplete + "%");
        }
    });
});

In this case, the only thing we are doing is get the percent complete each time the internal timer-scheduled poll returns a value and display it in a prepared span. That may look good, but What if I don’t like the add link or the upload link? Or what if I want change these texts?

Scenario 2 – Modifying the links ‘add’ and ‘upload’.
The last thing I want users to be is tied to defaults, let’s start by changing the texts for the actions add and upload. I’ll introduce you three new plugin parameters:
·         textAdd: Text for the link Add
·         textUpload: Text for the link Upload
·         textCancel: Text for the link Cancel
Sometimes we have to produce applications that are going to be viewed by people from different cultures and languages, so the static texts are not an option for these cases, and that’s why I let open the text for these links. In the example I use a simple way to change the texts, but I want you to realize the power of this choice.

@{
    // get the text from wherever i.e. some i18n service in the application
    var textAdd = "Add a file to upload list";
    var textUpload = "Start Upload";
    var textCancel = "Abort Operation";
}
(...)
<script type="text/javascript" defer="defer">
    $(function() {
        $("#file1").neatupload({
            postDataUrl: $("input[type=hidden][id=upload-action]").val(),
            fnShowStatus: function (plugin, index, data) {
                $('span#progress-file').text(data.PercentComplete + "%");
            },
            textAdd: "@textAdd",
            textUpload: "@textUpload",
            textCancel: "@textCancel"
        });
    });
</script>

Another way for changing these links and not just the texts is through other two parameters, this time they replace the entire ‘a’ element by whatever you specify to. They are:
·         selectorButtonAdd: jQuery selector indicating how to get the DOM element that will click for the add action.
·         selectorButtonUpload: jQuery selector indicating how to get the DOM element that will click for the upload action.

This time the parameters regarding the text for add and upload will be ignored, if you change the element, you are responsible for give them their own text using any strategy. So the example is as follows:
@{
    // get the text from wherever i.e. some i18n service in the application
    var textAdd = "Add a file to upload list";
    var textUpload = "Start Upload";
    var textCancel = "Abort Operation";
}
(...)
<script type="text/javascript" defer="defer">
    $(function() {
        $("#file1").neatupload({
            postDataUrl: $("input[type=hidden][id=upload-action]").val(),
            fnShowStatus: function (plugin, index, data) {
                $('span#progress-file').text(data.PercentComplete + "%");
            },
            textCancel: "@textCancel",
            selectorButtonAdd: "input[name=addfile]",
            selectorButtonUpload: "input[name=uploadfile]"
        });
    });
</script>
(...)
<input type="button" name="addfile" value="@textAdd" />
<input type="button" name="uploadfile" value="@textUpload" />

The source code is available here.