A site devoted to discussing techniques that promote quality and ethical practices in software development.

Thursday, June 2, 2011

Caveat in using Matlab DLL in ASP.Net

I was crafting another WCF Service using basicHttpBinding that used a Matlab NE Builder generated DLL and was keep getting System.TypeLoadException on the class' constructor. This problem was reported but there was no conclusive explanation other than ASP.Net page time out being set too short. But setting that to a long value did not help and certainly did not help me.

My unit test using the DLL directly proved that the DLL was working perfectly and hence it must be environmental. Not the person easily defeated by this kind of issue, I took this up as just another challenge. There must be an explanation.

Being certain that it was environmental, I began to investigate the ASP.Net runtime infrastructure. I know when the Matlab DLL, called a deployment DLL for all the right reasons, was first used, Matlab runtime will expand the embedded scripts, albeit compiled, into a runtime folder structure under the matlab_mcr folder. This folder structure replicates the same directory structure from the root directory when the deployment DLL was built. matlab_mcr is the folder created at a nominated directory.

So if when you build your deployment DLL in say
"C:\Documents and Settings\John\My Documents\My Matlab Project"

Matlab runtime will try to create a structure like this:
[TempDirBase]\matlab_mcr\Documents and Settings\John\My Documents\My Matlab Project

where [TempDirBase] is the base directory where your run time is going to use. If it is the %Temp%, then in XP it is
"C:\Documents and Settings\John\Local Settings\Temp"

If you are like me using very verbose and easy to remember project name organized in a meaningful directory structure and using long matlab function names, the path that Matlab replicates at run time can easily approach the maximum limit of a path name.

In fact the cause of the TypeLoadException by the matlab class was exactly what I encountered when I was using NUnit to unit test code that used a deployment DLL. In NUnit I simply changed the shadow copy directory.

In the ASP.Net situation, the length of the default ASP.Net temporary directory path coupled with my build location caused the operating system to fail to recreate the matlab run time structure because the path length exceeded the maximum permissible length.

Armed with prior experience and the ability to change the ASP.Net default temporary file directory to a different location, I used a 2-prong attack to ensure that matlab's silly recreation of the directory structure will not bother me anymore.

Here is my technique to fix this problem:
1) Build the deployment DLL in a extremely cryptic and shorten path.
Build the deployment DLL in a very cryptic and extremely short path like this c:\0\1 or c:\01. I choose this pattern because I am going to have several of these deployment DLLs and so each one can grab a different number as directory name. Since this is only for run time and it is totally meaningless human, they can be very cryptic. The aim is to make it as short and as flat as one can get and.spare no mercy to Matlab.

You can even use these numbers as your extra version identifier.

This recommendation does not mean that you need to store your project that way. You should store them in as meaningful directory as you can for easy of maintenance and team spirit. By default action, when Matlab replicates this at runtime, it discloses your project organization and if you store your project in the profile area, Matlab can disclose who you are. By using a cryptic and short build path, you prevent Matlab from disclosing such information.

Since I always build these deployment DLLs with a batch file for good technical reasons, it is only a matter of adding extra batch commands to create the build directory described above, and to copy all the .m, .prj, .cs, etc. to the build directory and kick the build from that build directory.

When the build is finished, you copy the output files back to a generated output directory in my solution. The batch file can include commands to remove the build directory.

2) As an extra measure to combat this Matlab bloated directory problem, you change the ASP.Net temporary file directory, which is conveniently applicable per application using the web.config.

This can be achieved by specifying your temporary directory to a very short path name in the tempDirectory attribute of the <compilation> element.

Arm with these and a very long "ASP Script timeout" for your application, changed via the IIS console, you should have no trouble with using Matlab deployment DLL in IIS.

4 comments:

Anonymous said...

I have a matlab dll built from builder ne. I want to use in in a asp.net page but I'm unable to do so.
Please guide me.

Anonymous said...

Nice! Fixed my problem! Thanks!

Roger Breton said...

Thank you for sharing your hard-earned experience with Matlab and ASP.NET and IIS.

I am about to embark on a similar path, using C#, Matlab Assemblies and my ASP.NET web hosting provider.

I have been burned by using of a COM DLL which works perfectly fine on my Win2k8 R2 Server / IIS7 at home but, for some strange reason, fails to load on my web hosting provider Wiin2k8 R2 / IIS7 system.

I am trying to do my homework in advance of getting too far involved in this approach with using Matlab in my web application in case I encounter yet another mishap that my web hosting provider will have no clue about.

One question, if I may. So far, I’m pretty clear on the compiling of the .NET Assembly step and adding References to the Web Application from within Visual Studio. But what about the actual .dll themselves, myCompiledMatlabFunctions.dll and MWArray.dll, can they simply be dumped into the bin directory? Will ASP actually find them there? (Please don’t mind my unusually long filename above…)

Last but not least, about the MCR itself. I understand nothing will happen until my hosting provider agrees to install it on their server? With the rights as IIS7, IIUSRS account or Network Service account?

/ Roger Breton

L. Mar said...

Hi Bretton,

"I have been burned by using of a COM DLL which works perfectly fine on my Win2k8 R2 Server / IIS7 at home but, for some strange reason, fails to load on my web hosting provider Wiin2k8 R2 / IIS7 system. "
Yes, this has been the most challenging task in our project. The project was conceived and tested in IIS5 and when it was about to roll out, IIS6 was used and then IIS7. The gradual tightening of security created all sort of failures - not from neglects in our code. Since then I have left the project and has not dealt with IIS7. But perhaps the steps I took to get it going may give you some idea of how to diagnose the problems - which typically is security related when dealing with IIS. The tool that got me out of the trouble is called Process Monitor http://technet.microsoft.com/en-us/sysinternals/bb896645, lots of patience pouring through logs and good understanding of underlying technology. Going back to my notes Matlab NE Builder's product does not use COM and relies on .Net integration with your IIS application. However, it does use Java deep down.

"But what about the actual .dll themselves, myCompiledMatlabFunctions.dll and MWArray.dll, can they simply be dumped into the bin directory? Will ASP actually find them there?"
It all depends on the deployment. If you deploy the Matlab runtime using Matlab provided installation, the MWArray.dll is, from memory, deposited into the GAC. Hence you need not worry except to ensure that the deployed assembly matches the one you use in development.

The ASP.Net supporting assembly must therefore, like with or without using Matlab, be deployed into the web application directories. Those assemblies must also be in strongly named. This is the reason why I spent so much effort to find a way to stamp a meaningful version number into NE Builder produced assembly. It is disappointing to see Matlab failing to appreciate this point, even when it was brought to their attention. Hope the new Matlab studio would have addressed this.

"about the MCR itself. I understand nothing will happen until my hosting provider agrees to install it on their server? With the rights as IIS7, IIUSRS account or Network Service account?"
Yes, MCR needs to be installed as part of your IIS application deployment. Having said that, you then need to watch where Matlab runtime expands your scripts to. Failure to deal with that can cause runtime failure, even when all the security aspects are taken care properly.

From memory, in our project we did not change the account type in the IIS application. We used whatever is the standard recommendation. May be "Network Service Account". Use Process Monitor to see what goes wrong if you are in trouble.

For this kind of development, it is best to use a virtual machine as that can allow you to test your server application afresh. Good luck.

One final note, I vaguely remember that MCR requiring write permission to some folder of the version number associated with the MCR. It does not write anything in it but demanding the write permission, which Network Service Account does not have. Hence we had to give it the permission required and we had not yet determined if such relaxation representing a security risk.

Unfortunately I cannot locate my notes on this issue nor publishing it in my blog. My advice is if confronted with failure to run (usually the first time on a clean machine), use Process Monitor to look for "Access Denied" failure.

As you can see, deploying Matlab to IIS is very troublesome and depending on your project requirement, it could be a strong influence to the successful adoption of your product.

Blog Archive