Very slow UpdatePanel refresh when containing big ListBoxes or DropDownLists
Update: this fix is now on Github: Github. Get the latest version from there.
The scenario is pretty straightforward: a ListBox or DropDownList or any control that renders as a Select html element with a few thousand entries or more causes an asynchronous UpdatePanel update to become incredibly slow on Internet Explorer and reasonably slow on FireFox, keeping the CPU to 100% during this time. Why is that?
Delving into the UpdatePanel inner workings one can see that the actual update is done through an _updatePanel Javascript function. It contains three major parts: it runs all dispose scripts for the update panel, then it executes _destroyTree(element) and then sets element.innerHTML to whatever content it contains. Amazingly enough, the slow part comes from the _destroyTree function. It recursively takes all html elements in an UpdatePanel div and tries to dispose them, their associated controls and their associated behaviours. I don't know why it takes so long with select elements, all I can tell you is that childNodes contains all the options of a select and thus the script tries to dispose every one of them, but it is mostly an IE DOM issue.
What is the solution? Enter the ScriptManager.RegisterDispose method. It registers dispose Javascript scripts for any control during UpdatePanel refresh or delete. Remember the first part of _updatePanel? So if you add a script that clears all the useless options of the select on dispose, you get instantaneous update!
First attempt: I used select.options.length=0;. I realized that on Internet Explorer it took just as much to clear the options as it took to dispose them in the _destroyTree function. The only way I could make it work instantly is with select.parentNode.removeChild(select). Of course, that means that the actual selection would be lost, so something more complicated was needed if I wanted to preserve the selection in the ListBox.
Second attempt: I would dynamically create another select, with the same id and name as the target select element, but then I would populate it only with the selected options from the target, then use replaceChild to make the switch. This worked fine, but I wanted something a little better, because I would have the same issue trying to dynamically create a select with a few thousand items.
Third attempt: I would dynamically create a hidden input with the same id and name as the target select, then I would set its value to the comma separated list of the values of the selected options in the target select element. That should have solved all problems, but somehow it didn't. When selecting 10000 items and updating the UpdatePanel, it took about 5 seconds to replace the select with the hidden field, but then it took minutes again to recreate the updatePanel!
Here is the piece of code that fixes most of the issues so far:
/// <summary>
/// Use it in Page_Load.
/// lbTest is a ListBox with 10000 items
/// updMain is the UpdatePanel in which it resides
/// </summary>
private void RegisterScript()
{
string script =
string.Format(@"
var select=document.getElementById('{0}');
if (select) {{
// first attempt
//select.parentNode.removeChild(select);
// second attempt
// var stub=document.createElement('select');
// stub.id=select.id;
// for (var i=0; i<select.options.length; i++)
// if (select.options[i].selected) {{
// var op=new Option(select.options[i].text,select.options[i].value);
// op.selected=true;
// stub.options[stub.options.length]=op;
// }}
// select.parentNode.replaceChild(stub,select);
// third attempt
var stub=document.createElement('input');
stub.type='hidden';
stub.id=select.id;
stub.name=select.name;
stub._behaviors=select._behaviors;
var val=new Array();
for (var i=0; i<select.options.length; i++)
if (select.options[i].selected) {{
val[val.length]=select.options[i].value;
}}
stub.value=val.join(',');
select.parentNode.replaceChild(stub,select);
}};",
lbTest.ClientID);
ScriptManager sm = ScriptManager.GetCurrent(this);
if (sm != null) sm.RegisterDispose(lbTest, script);
}
What made the whole thing be still slow was the initialization of the page after the UpdatePanel updated. It goes all the way to the WebForms.js file embedded in the System.Web.dll (NOT System.Web.Extensions.dll), so part of the .NET framework. What it does it take all the elements of the html form (for selects it takes all selected options) and adds them to the list of postbacked controls within the WebForm_InitCallback javascript function.
The code looks like this:
if (tagName == "select") {
var selectCount = element.options.length;
for (var j = 0; j < selectCount; j++) {
var selectChild = element.options[j];
if (selectChild.selected == true) {
WebForm_InitCallbackAddField(element.name, element.value);
}
}
} function WebForm_InitCallbackAddField(name, value) {
var nameValue = new Object();
nameValue.name = name;
nameValue.value = value;
__theFormPostCollection[__theFormPostCollection.length] = nameValue;
__theFormPostData += name + "=" + WebForm_EncodeCallback(value) + "&";
} That is funny enough, because __theFormPostCollection is only used to simulate a postback by adding a hidden input for each of the collection's items to a xmlRequestFrame (just like my code above) in the function WebForm_DoCallback which in turn is called only in the GetCallbackEventReference(string target, string argument, string clientCallback, string context, string clientErrorCallback, bool useAsync) method of the ClientScriptManager which in turn is only used in rarely used scenarios with the own mechanism of javascript callbacks of GridViews, DetailViews and TreeViews. And that is it!! The incredible delay in this javascript code comes from a useless piece of code! The whole WebForm_InitCallback function is useless most of the time! So I added this little bit of code to the RegisterScript method and it all went marvelously fast: 10 seconds for 10000 selected items.
string script = @"WebForm_InitCallback=function() {};";
ScriptManager.RegisterStartupScript(this, GetType(), "removeWebForm_InitCallback", script, true);
And that is it! Problem solved.
Comments
Thanks, Asibin :) you should have asked me this in Silver Church >:-) Anyway, the fix is only for list controls. That includes the DropDownList and ListBox controls, which on the html end are just two instances of the same element type: _select_. I am having trouble understanding how would you see this working for a GridView, which is essentially a display control, not an input control. You see, the fix works by simplifying the DOM right before the ASP.Net parsing of it, replacing a lot of <option> elements inside <select>s with a single hidden field. If you have multiple input fields or buttons, they each have their own element, their own ID; this fix wouldn't work. For large editable grids, with many input controls in them, consider using paging. If you are worried about the performance of getting a complete data source for each page or of caching it in the view state, try using custom paging and getting the data source for each page, like here: http://siderite.dev/2007/08/changing-gridview-pagecount.html. Also, you might consider using a server side view state, so as to minimize the transfer size of pages like here: http://siderite.dev/2007/04/saving-page-viewstate-out-of-page.html Hope that helps.
SideriteSiderite - really amazing work on this updatepanel thing and I love your blog on Coma lol. I have a question about this updatepanel fix. I downloaded the codeplex version and installed it. It is working for dropdowns and listbox, but can I just apply this to GridView and buttons etc? I noticed a few discussions on that topic and I tried it, but I do not seem to get a performance increase using this on GridView / Buttons within an updatepanel. Any help would be huge at this point because i'm kind of stuck with the updatepanels. Thanks
AsibinHi, I tried moving of list items even using jQuery. But even that hangs for 40000 items.
VenAssume you have small names for the options in the select element that is rendered, let's say 5 to 10 characters. Then, for each of the 50000 items, you will have something like <option value="12345">~8 chars</option>, so about 40 characters each. That is 2MB of data for the list alone, but in order to send the selection of items, more data is stored in the ViewState and also passed through the POST request. So more than 2MB of data at every action you perform! My guess is that the slow performance you were reporting was happening on your local machine. Now imagine someone would try to access that page on a slow net connection.
SideriteHi, 46000 items may be too big, but there are reasons why I had to manage the movement at server side. The screen has so many controls, which do post back, but I have to retain the values (items, selection) in both the listboxes, which will be a big pain at client side.
VenI just have to start with "Why so many?". It is clearly not a pleasant user experience to manually sift through 46000 items. Also, from what you are saying, it seems that you are controlling the selection and movement of items from one side to the other via server calls. If that is so, it is clearly a mistake. So, first advice is to make sure the only server calls are when loading the page and when saving the changes you make, the rest being done via Javascript. Then, in order to gauge the effect of the bug I was trying to fix with this post, set ScriptManager.EnablePartialRendering="False" or remove the updatepanels around the user control and see if it takes longer with the UpdatePanel than without it. If so, they you need to use my fix or something similar. Otherwise, you have a different problem. Good luck!
SideriteHi, I have an aspx page, which hosts a User Control within an update panel. This user control has 2 Listboxes with asp buttons to move items from one list of another and vice versa. The first list is getting around 46000 items. After the screen loads completely, even if I make a selection of items in the first list, the screen hangs. I do have your code implemented, but no result. Can you please help?
VenAnything I post on this blog, unless specified otherwise, is completely devoid of any lawyeresque bullshit. If you want to use it in any way, I am ok with it. It would be nice to reference me if using my work, though :) Second, the code replaces a html select element with a html hidden field element on submit. It doesn't affect the server code in any significant way. So no security issues that I can see.
SideriteWonderful work. I do have two questions: 1, is this open source Freeware-style, and 2, would this affect security issues in ANY unforseen way?
LazamatazI think, the Adapter (and fix-script by itself too) didn’t work for me, because for every list control it has fix-script only for itself (getElementById), not for all list controls (getElementByTagName). For me it wasn’t enough to call registerFix for every list control – I also needed to fix every ‘select’-tag control in the fix-script by working with getElementByTagName. And yes – I’m disabling the callback only once – in the EnsurePanelFix-method. As for position in Page_load – I fill my list controls there, so I call registerFix after that filling (when the content is formed).
ilataWell, normally the disabling of the callback should have been called only once anyway. It is strange that the position of the method in Page_Load is important. EnsurePanelFix seems to be mainly a Sharepoint specific method. All in all I am glad it works, but it may be that it does for all the wrong reasons :) Also I haven't tested the adapter in a Sharepoint scenario, but I believe that it should have worked. I am curious on why it didn't work, but I lack the time to investigate further.
SideriteSiderite, First I’ve tried to use Adapter, but my UpdatePanel continued to refresh slow. Then I returned to my original code and applied all Your suggestions. But once again–it didn’t work for me. I checked–the only list control, for which fix-script worked–was the first dropdownlist–no matter which control caused postback. Then I brought back getting of all the ‘select’-tag controls in the fix-script–and also in my registerFix-method I added "Items.Count > 0"-check - and it began to work like it worked before–only from the second postback. Though there were dropdownlists as direct children of page – at the first postback it still didn’t work. Then I did next: 1) moved calling registerFix to the end of Page_Load (when all the filling-methods have finished their work), 2) moved calling “WebForm_InitCallback=function() {};” from the fix-method to the EnsurePanelFix-method, which is called in Page_Load of the parent usercontrol of UCDefault and it is fixing somehow UpdatePanel postback (this method is from Internet): // Fix for the UpdatePanel postback behaviour static public void EnsurePanelFix(Type type, Page page) { if (page.Form != null) { String fixupScript = @"_spBodyOnLoadFunctionNames.push(""_initFormActionAjax""); function _initFormActionAjax() { if (_spEscapedFormAction == document.forms[0].action) { document.forms[0]._initialAction = document.forms[0].action; } } var RestoreToOriginalFormActionCore = RestoreToOriginalFormAction; RestoreToOriginalFormAction = function() { if (_spOriginalFormAction != null) { RestoreToOriginalFormActionCore(); document.forms[0]._initialAction = document.forms[0].action; } } WebForm_InitCallback=function() {};"; ScriptManager.RegisterStartupScript(page, type, "UpdatePanelFixup", fixupScript, true); } } Each step by itself didn’t fix the first-postback problem, but both of them somehow made it. And now my UpdatePanel refreshes fast on any postback of any control. Siderite, thanks a lot for Your help! Hope, I did right steps in my solution :) p.s. If You have some questions or suggestions – I’ll be pleased to contribute.
ilataYesrm
AnonymousFirst of all I would like to point out some code improvements: - instead of "if (this.Controls[i].GetType().Name == "DropDownList") { RegisterScript(this.Controls[i] as DropDownList); }" use "DropDownList ddl=this.Controls[i] as DropDownList; if(ddl!=null){ RegisterScript(ddl); }" - the code "for (int i = 0; i < this.Controls.Count; i++)" only gets the top controls in the Page. If you place all dropdowns in a Panel, for example, it will not fire once. Why haven't you used the Adapter approach? - the javascript code you are using has a getElementsByTagName('select') so it itterates through all selects on the page. Since the RegisterDispose method needs a control to be disposed as a parameter, I suggest you use getElementById(ddl.ClientID). getElementsByTagName is very slow. So, if you don't want to use the adapter from codeplex directly, try this code: private void registerFix(Control parent) { foreach(Control child in parent.Controls) { if (child is ListBox || child is DropDownList) { fix(child); } else { registerFix(child); } } } private void fix(Control control) { string script=string.Format(@"var select=document.getElementById('{0}'); if (select) {{ var stub=document.createElement('input'); stub.type='hidden'; stub.id=select.id; stub.name=select.name; stub._behaviors=select._behaviors; var val=new Array(); for (var i=0; i<select.options.length; i++) {{ if (select.options[i].selected) {{ val[val.length]=select.options[i].value; }} }} stub.value=val.join(','); select.parentNode.replaceChild(stub,select); }}",control.ClientID); ScriptManager sm = ScriptManager.GetCurrent(this.Page); if (sm != null) sm.RegisterDispose(updControl, script); } run registerFix(Page) in Page_Load. Now, about why your original code doesn't work unless you postback once, I don't know. It may be because only from the second postback there is a dropdownlist as a direct child of page, or maybe some other reason. Try the code above and tell me if you experience the same behaviour.
SideriteHello, Siderite. I’ve applied Your fix to my solution in SharePoint. My solution has a system masterpage (I don’t change there anything), a page (Default.aspx) and a usercontrol (UCDefault.ascx). In my page I added ToolkitScriptManager and UpdatePanel, and UpdatePanel has a usercontol object UCDefault1 inside it. In my usercontrol I have lot’s of DropDownLists with thousands of items. So, on Page_Load event of usercontrol I make next: protected void Page_Load(object sender, EventArgs e) { for (int i = 0; i < this.Controls.Count; i++) { if (this.Controls[i].GetType().Name == "DropDownList") { RegisterScript(this.Controls[i] as DropDownList); } } } And in RegisterScript (which was made from Your code) I do next: private void RegisterScript(DropDownList updControl) { string script = string.Format(@" var all_selects = document.getElementsByTagName('select'); if (all_selects) {{ for (var j=0; j < all_selects.length; j++) {{ var select = all_selects[j]; if (select) {{ var stub=document.createElement('input'); stub.type='hidden'; stub.id=select.id; stub.name=select.name; stub._behaviors=select._behaviors; var val=new Array(); for (var i=0; i<select.options.length; i++) if (select.options[i].selected) {{ val[val.length]=select.options[i].value; }} stub.value=val.join(','); select.parentNode.replaceChild(stub,select); }}}}}};", updControl.ClientID); ScriptManager sm = ScriptManager.GetCurrent(this.Page); if (sm != null) sm.RegisterDispose(updControl, script); script = @"WebForm_InitCallback=function() {};"; ScriptManager.RegisterStartupScript(this, this.GetType(), "removeWebForm_InitCallback", script, true); } As You see, in script I get all the DropDownLists and do all the steps from your code for every DropDownList, I’ve found. As a result, UpdatePanel refreshes fast, but beginning from the second postback. I don’t understand, why it doesn’t work for the first postback. Can You help me, please?
ilataI remember specifically removing the Callback functionality for a good reason that, alas, I cannot remember :) So, no, at the moment there is no workaround for it. I would ask you, though, to please leave some info on how to do it, if you ever get around to making it work.
SideriteI have tried the solution and its rocking. But still i have a problem,we are using ICallBackEventHandler to autosave the data in form and when i removed WebForm_InitCallBack, its not working. Is there any workaround for this?
AnonymousProblem Solved! This solution only implements itself if the list box contains more than 50 items (which is fine for most cases) If you are like me and want to start with an empty list box, then edit ListControlUpdatePanelFixAdapter.csand change the MinItemCount property to return -1 (instead of 50). Then rebuild the solution and grab the new output dll. Now this solution always implements itself and no more slow listboxes
JedGreat post - works well - thanks for your work. One problem though is that it only works if the large list item is loaded in the PageLoad event. If the list box is loaded via an update panel, then the problem returns, but only for the first post back - then it is fixed again. I'll try and decipher the code a bit more, but if anyone has solved that issue I would love to know more Cheers
JedThank you, thank you, thank you. This problem was driving me crazy. Your fix worked like a charm.
AnonymousWell, I hardly remember it now, but one of the major things one must do in order to identify ASP.Net problems is to use Reflector and the File Dissassembler addon to get all the sources for the .Net libraries, especially, in ASP.Net's case System.Web and System.Web.Extensions. You also get the javascript files that are used, so you can browse through them. One other hint is to use a debugger. Register a _debugger;_ keyword as a submit statement, then debug with whatever debugger works best. For example, at the time, I had installed Microsoft's Script debugger, which, even if it was pretty basic, succesfully found the js source at the debug point as opposed to Visual Studio. So, the bottom line is that you need to think like a computer :)
Sideritemany thanks for that solutions man! could you tell me how do you investigate the root cause? I mean what tools or debugger you used to identify the problem? many thanks!
henryThanks for this Siderite. I'll be honest and say I have no idea how it fixes the problem but I simply copied your code, used a C# to VB converter, had to change 1 minor part of it and it works like a charm. Lovely.
Chris Westgr8 post!!!1
sj26This is marvellous. It helped as to solve a critical issue
RameshPerfect, solved a very irritating problem for me too.
G0ggyThe AjaxControlToolkit cascading control uses web services to get the data and populate the dropdown lists. This fix of mine works exclusively on UpdatePanel postbacks.
Sideritehi, can this solution applied to Ajaxtoolkit cascadingdropdown as well?
mavenDon't usually post in these forums, but this solution worked a treat and saved me a lot of hassle. I use it for a number of Drop Down Lists I have on my page and it works by simply passing the DDL to the RegisterScript function as Siderite mentioned in the comments. Thanks Siderite!
EuanThank you for your excellent research! I'm impressed!!
Joel McClureWell, the ListBox is a glorified DropDownList. It should work for both, I don't see why not. You are welcome.
SideriteThanks very much Siderite!
KimI have posted a solution that works for both here: http://forums.asp.net/p/1360932/2832582.aspx#2832582 Thanks again, Siderite!
Chris in Wilmington, NCThis looks exactly like the problem I'm having with my dropdown lists, they have around 1000 rows, due to business requirements I cant make them any smaller. On initial load, the performance is ok, but whenever the user chooses an item in the list, the refresh takes much longer. Sorry for the stupid question, but your sample code is for a listbox, would anything need to be changed to get it to work for a ASP.net ddl ?
KimWell, you would have to tell me more and I would have to have more time to investigate [Hint! :)]. Try profiling the javascript, try it in FireFox and Chrome and see if the javascript or the rendering are at fault, etc.
SideriteI was able to implement your solution on a standalone page. I am trying to implement it on some dropdown list controls that are in the Edit- and InsertItemTemplates of a FormView which is in a Content Page. The ScriptManager is in the Master page. The scripts appear to be firing (I stuck alerts in them) but the performance doesn't improve. Do you have any suggestions that I could check? Thank you and Merry Christmas.
Chris in Wilmington, NCYou're welcome! Someone making use of what I write fuels this blog.
SideriteThank you. Thank you. Thank you! I implemented the solution via VB. I registered the element swap part of the script to the RegisterDispose event. I am collecting the selected ListBox items via the button's OnClientClick event so that I can process and display them before the PostBack.
Chris in Wilmington, NCI've got the vb equivalent if anyone's interested. I just can't seem to post it to this blog. It keeps saying Your HTML cannot be accepted.
AnonymousGreat solution. Do you have a vb equivalent?
AnonymousSo, here is the entire method, with a listbox as a parameter. Use it in Page_Load: /// <summary> /// Use it in Page_Load. /// lb is any ListBox /// </summary> private void RegisterScript(ListBox lb) { string script = string.Format(@" var select=$get('{0}'); if (select) {{ var stub=document.createElement('input'); stub.type='hidden'; stub.id=select.id; stub.name=select.name; stub._behaviors=select._behaviors; var val=new Array(); for (var i=0; i<select.options.length; i++) if (select.options[i].selected) {{ val[val.length]=select.options[i].value; }} stub.value=val.join(','); select.parentNode.replaceChild(stub,select); }}; ",lb.ClientID); ScriptManager sm = ScriptManager.GetCurrent(this); if (sm != null) sm.RegisterDispose(lb, script); string script = @"WebForm_InitCallback=function() {};"; ScriptManager.RegisterStartupScript(this, GetType(), "removeWebForm_InitCallback", script, true); }
SideriteHow would you do this using vb?
AnonymousYes i changed lptest to receive it as an parameter and then first called callbackdisposble and then called registerscript to accept the dropdownlist as an parameter, still i see it takes lot of time, can you please let me know how to fix this Thanks
AnonymousThe method in the post is a sample only. If you want it to work for more dropdownlists you need to change it to accept lbTest as a parameter.
Sideritewhen i call registerscript for both the dropdownlist, i am getting error 'Select.options.length' is null or not an object but works fine if i have only one dropdownlist,
AnonymousDear Sir, Thank you so much, it works for me.
AnonymousWell, you would have to take the last piece of code, the one that disables the old style CallBack, and put it in a separate method that you execute once. Then add a parameter to RegisterScript (maybe rename it to something else) that would be the ListBox that you want processed and use that ListBox instead of lbTest. And for the end, load the CallBackDisable method then RegisterScript for each of the listboxes you need fixed in the Page_Load event.
SideriteWould this work if I have mulltiple ddls that I need to apply it against and if so - how?
AnonymousThank you so much for this quick fix. You literally saved my life.
WeiI was right. The array in question is called _behaviors (with American spelling). I've updated the code with one line, the one with _behaviors in it :) Thank you for pointing out the problem.
SideriteAs far as I see, that error is thrown by Application.addComponent when it finds in the array _components a component with the same id as the one to be added. If indeed my code is to blame, then possibly Ajax attaches to elements an array of extenders that it clears onsubmit, then it reattaches them. I will investigate.
SideriteI have tried your solution and it worked fine. However once I add any extender to the dropdownlist such as ListSearchExtender it gave an error "Sys.InvalidOperationException: Two component with the same id 'ListSearchExtender1' can't be added to the application". Could you tell me how to deal with this error? Thanks a lot.
WeiWow, thanks a lot! I've searching for both a reason and a solution for this unusual problem, and you've shown me both! What a marvelous post!
Anonymous