13 September 2012
It's got to the point with a Web Service I'm developing at work to run it through some stress tests and try to work out what kind of load we can handle with the hardware we have in mind for its hosting environment. We also have an arbitrary concept of the load we'd like it to be able to handle, so we need to see if the hardware solution we have in mind is even in the right ballpark.
It's a WCF Web Service running against .Net 4.0 but it's going to be hosted in IIS 6; the Web Servers our Infrastructure Team provide us with are all Server 2003 and although there are mid-term plans to start upgrading to 2008 and IIS 7 this isn't going to happen before this service needs to be live.
The Web Service effectively backs on to a Data Service we already have for our own private use that handles querying and caching of data and metadata, so this Web Service's primary job is to expose this data in a saner manner than the internal interface (which was built some years back and has a lot of legacy features, exposes too much craziness from the backing database which evolved over the best part of a decade and the internal service is communicates over .Net Remoting as it was originally written against .Net 2.0!). So all we're really doing is translating requests from the Web Service model into the internal format and then performing a similar act on the responses, there won't be too many caching concerns, any direct db access or much file IO; the bulk of its time I expect will be waiting on the internal service.
To kick things off, I wanted to get an idea of how much load the internal service could handle on a single server and then see how things changed when I put a Web Server in front of it, hosting the new Service. To cut a long story short, I was able to reproduce requests to the internal service from the busiest few hours the night before and serialise them all to disk so that they could be replayed at any time against the stress-testing environment. The optimistic plan being to ramp up the speed at which the requests are replayed and see where the bottlenecks start appearing and at what sort of loads!
The first problem I encountered was when replaying these from a test app on my computer to the data service in the load test environment. As the Web Service requests were ramping up the request times were getting longer and longer. I looked on the Web Server and found that it was doing very little; CPU and memory use was low and looking at the "Web Service"* performance counters it was not only dealing with very few requests but there weren't any queuing up which had been my initial thought.
* (The "Web Service" in this context relates to the service in IIS, it's not a Web Service in the context of an ASP.Net Web Service, for example).
Some googling revealed that outgoing http requests are limited by default in .Net applications, so the requests were getting queued up locally and weren't even getting to the server! It seemed like only two concurrent requests were being handled though I'm sure documentation suggested that it should have been 16. Not that it matters, really, that would have way been too low as well!
Adding this to the app.config sorted it out:
<system.net>
<connectionManagement>
<add address="*" maxconnection="65535" />
</connectionManagement>
</system.net>
The "*" means that this will be applied to any endpoint. The 65535 was what I found in the example and I've been struggling to find a definitive answer as to whether this is the maximum value that it can be or not! :S
So now I was able to overload the server with more concurrent requests than it could handle at once and they were getting queued up. The performance counters are just invaluable here; there are some available for the "Web Service" - which tells us about what's happening at the IIS level - and for "ASP.Net" - which tells us about what's getting into .Net framework. So there could theoretically be limits in IIS that need to be overcome and/or limits in ASP.Net, keeping an eye on these counters gives an insight into where the bottlenecks might be.
I ramped up the request replay rate and it seemed like the maximum concurrent connections that I could achieve was around 250. The CPU load was low, around 15%, and the memory usage was fairly meagre. Most of the time in the requests was waiting for the data service to respond, so threads were just hanging around. Not enough threads for my liking! I wanted to use more of the resources on the box and be able to deal with more requests.
I spent aaaages trying to find information about whether this 250 was a limit set in IIS somewhere or if it was somehow related to the default maximum of 250 threads per CPU in a .Net ThreadPool or just where to find the knob in IIS that I can turn up to eleven! And my conclusion is that it's very difficult to find a definitive resource; so much of the information is for different versions of IIS (5 or 7, usually) and they usually do not make clear which settings are applicable to which versions and which changes I could expect to help me.
So I'm just going to cut to the chase here. For IIS 6 (six!) I needed to do two things.
Firstly, edit the machine.config for applicable ASP.Net version (I'm running against 4.0 and IIS is currently set up to run as 32-bit since to enable 64-bit I believe that all of the applications must be 64-bit and where we're going to host this Web Service I can't be sure that this will be the case):
<system.web>
<processModel maxWorkerThreads="100" maxIoThreads="100" minWorkerThreads="50"/>
"100" is the maximum value that maxWorkerThreads and maxIoThreads can take. "50" was the value I found in the examples I encountered, presumably this lower bound is set so t hat bursty behaviour is more easily handled as a base number of threads are always spun up (starting up a load of threads is expensive and if a patch of high traffic hits the service then having to do this can result in requests being queued). These were the settings for which there is so much apparently contradictory information out there; some people will tell you the default values are 50 or 12 x the number of CPUs or 5000 depending upon the version of IIS (but infrequently saying which version they were talking about) or the version of ASP.Net (but infrequently saying which version they were talking about!) or.. I don't know.. what day of the week it is!
The second was to increase the number of processes that were used by the Web Service's App Pool. This is done by going into the Performance tab on the App Pool's properties page and increasing the "Maximum number of worker processes" upwards from 1.
There are considerations to bear in mind with increasing the number of worker processes; the first that often jumps out in documentation is issues with Session - any Session data must be maintained in a manner such that multiple processes can access it, since requests from the same Client may not always be handled by the same process, and so an in-process Session would be no good. The Service I was working with had no need for maintaining Session data, so I could happily ignore this entirely.
With these changes, I was able to ramp up the requests until I was max'ing out the CPU in a variety of circumstances; I set up different tests ranging from requests that could be completed very quickly (within, say 20ms) to requests that would artificially delay the response to see how concurrent requests would build up. Without trying too hard I was able to get around 500 requests / second with the quick responses and handle 4000 concurrent requests happily enough with the long-running ones.
This is hardly crazy internet-scale traffic, but it's well in excess of the targets that I was originally trying to see if I could handle. And since the "hardware" is a virtualised box with two cores I'm fairly happy with how it's working out!
Something we have to consider in exposing this data to external Clients is just how much data we may be passing around. The internal Data Service that is essentially be wrapped is just that; internal. This means that not only will all the data be passed around internal network connections to the requesting Web Server (that will use that data in rendering the site) but also that we can format that data however we like (ie. binary rather than xml) and take all sorts of crazy shortcuts and liberties if we need to as we have to understand the data implicitly to work efficiently with it. For the Web Service, the data's going to be broadcast as xml and we want to make the compromise of exposing a consistent and predictable data model with a simple API. This means that we'll be dealing with much larger quantities of data and we have to pass them out of our private network, out into the real world where we have to pay for transfers.
This really struck home when, during the load tests, I was max'ing out the network utilisation of my connection to the server. My connection to the office network runs at 100Mbps (100 megabits, not megabytes) - other people get 1Gb but I get 100 meg, cos I'm really lucky :) This was in tests where a large response was being returned with a high rate of handled requests. To try to push these tests closer to the limits I had to borrow some other workstations and run requests through wcat (http://www.iis.net/downloads/community/2007/05/wcat-63-(x86)).
So I figured the obvious first step was to look into whether the response were being compressed and, if not, make them.
I examined a request in Fiddler. No "Accept-Encoding: gzip" to be found. A flash of recollection from the past reminded me of the EnableDecompression property available on the Client proxy generated by Web References. I set it to true before making the request and looked at the request in Fiddler.
Success! The http header for gzip support was passed.
But fail! It didn't make the slightest bit of difference to the response content.
Further googling implied that I was going to be in for some hurt configuring this on IIS 6. The good news is that WCF with .Net 4.0 would support compression of the data natively unlike 3.5 and earlier which required workarounds (I got as far as looking at this Stack Overflow question before being glad I didn't have to worry about it).
This blog post was a lifesaver: IIS Compression in IIS6.0.
To summarise:
After this, the response were being successfully compressed! As it's xml content it's fairly well compressible and I was getting the data trimmed down to approximately an 8th of the previous size. Another result!
Of course, there's always a compromise. And here it's the additional load on the CPU. For the sort of traffic we were looking to support, it seemed like it would be a good trade off; this CPU load overhead in exchange for the reduction in the size of the transmitted data.
Incidentally, Hanselman has a post here Enabling dynamic compression (gzip, deflate) for WCF Data Feeds, OData and other custom services in IIS7 illustrating just how much easier it is in IIS 7. If you've got a server hosted by that, good times! :)
I'm actually contemplating requiring gzip support for all requests made to the service to ensure we keep public traffic bandwidth in check. Since it's just an http header that specifies that the Client supports it, we can check for it from the Service code:
var acceptsGzip =
(WebOperationContext.Current.IncomingRequest.Headers["Accept-Encoding"] ?? "")
.Split(',')
.Select(v => v.Trim())
.Contains("gzip", StringComparer.InvariantCultureIgnoreCase);
And if we decide that Client gzip support is required then an error response indicating an invalid request can be returned if this header is not present.
This is probably going to sound really obvious but it caught me out when I did some testing with high numbers of concurrent requests; be careful how you deal with file locking when you have to read anything in!
I have a config file for the Service which has various settings for different API Keys. The data in this file is cached, the cache entry being invalidated when the file changes. So for the vast majority of requests, the cached data is returned. But when it's contents are changed (or the Service is restarted), the file's contents are read fresh from disk and if there are a multiple requests arriving at the same time there may well be simultaneous requests to that file.
I had previously been using the following approach to retrieve the file's content:
using (var stream = File.Open(filename, FileMode.Open, FileAccess.Read))
{
// Do stuff
}
but the default behaviour is to lock the file while it's being read, so if concurrent read requests are to be supported, the following form should be used:
using (var stream = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.Read))
{
// Do stuff
}
This way, the file can be opened simultaneously by multiple threads or processes. Hooray! :)
Posted at 22:56
12 August 2012
Following recently discovering how easy it is to implement Dependency Injection into ASP.Net MVC Controllers by rolling my own ControllerFactory I wanted to see if the same sort of thing could be done for some Web Services I'm working on - I've never seen any production code that does it but seemed like the sort of thing that should be possible!
"Discovering" in this case only meant googling for the extension points that I'd heard were built into MVC as standard; hardly the most earth-shattering of realisations! :) In case it saves anyone time, it's as simple as having an implementation of System.Web.Mvc.IControllerFactory and specifying this as the ControllerFactory to be used in the Application_Start method (in Global.asax) -
protected void Application_Start()
{
AreaRegistration.RegisterAllAreas();
RegisterRoutes(RouteTable.Routes);
ControllerBuilder.Current.SetControllerFactory(new ControllerFactory());
}
The most basic implementation runs along the lines of:
using System;
using System.Web.Mvc;
using System.Web.Routing;
using System.Web.SessionState;
namepsace Demo
{
public class ControllerFactory : IControllerFactory
{
public IController CreateController(
RequestContext requestContext,
string controllerName)
{
// Return appropriate controller here, using whatever factory class or dependency
// injection method is appropriate to the solution..
}
public SessionStateBehavior GetControllerSessionBehavior(
RequestContext requestContext,
string controllerName)
{
return SessionStateBehavior.Default;
}
public void ReleaseController(IController controller) { }
}
}
where requestContext exposes an HttpContext property so Cache, Request, Response and Server can be accessed.
All very straight forward (which made me wish I'd gotten around to looking into this long ago!).
I wasn't as confident that it would be as simple to construct an equivalent for a WCF Web Service as I think there was quite a culture shift from the Microsoft that created WCF for .Net 3.5 and that which wrote the MVC framework, that tried to make use of existing standards, that included jQuery with the framework and that released the framework code under an open source license! But it couldn't hurt to try!
In the Service's ServiceHost declaration a Factory may be specified which will be used to instantiate the Service type instead of the default being taken, which is to try to find a public parameter-less constructor and to call that.
<%@ ServiceHost
Language="C#"
Debug="true"
Service="Demo.ExampleWebService"
Factory="Demo.ExampleWebServiceHostFactory"
CodeBehind="ExampleWebService.svc.cs"
%>
The factory needs to inherit from the WebServiceHostFactory class. A given factory implementation may handle the instantiation of various Service types but for my purposes, and for the example here, I'm going to work on the basis that there is one factory per service (if different services are to be handled by a single factory then the "serviceType" argument passed around can be consulted to alter the behaviour as required).
The instantiation process follows theses steps:
Example code may make this clearer!
using System;
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Channels;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;
namespace Demo
{
public class ExampleWebServiceHostFactory : WebServiceHostFactory
{
protected override ServiceHost CreateServiceHost(
Type serviceType,
Uri[] baseAddresses)
{
return new ExampleWebServiceHost(serviceType, baseAddresses);
}
private class ExampleWebServiceHost : ServiceHost
{
public ExampleWebServiceHost(
Type serviceType,
params Uri[] baseAddresses) : base(serviceType, baseAddresses)
{
foreach (var cd in this.ImplementedContracts.Values)
{
cd.Behaviors.Add(new ExampleWebServiceProvider());
}
}
}
private class ExampleWebServiceProvider : IInstanceProvider, IContractBehavior
{
public object GetInstance(InstanceContext instanceContext)
{
// TODO: Return service instance here
}
public object GetInstance(InstanceContext instanceContext, Message message)
{
return this.GetInstance(instanceContext);
}
public void ReleaseInstance(InstanceContext instanceContext, object instance) { }
public void AddBindingParameters(
ContractDescription contractDescription,
ServiceEndpoint endpoint,
BindingParameterCollection bindingParameters) { }
public void ApplyClientBehavior(
ContractDescription contractDescription,
ServiceEndpoint endpoint,
ClientRuntime clientRuntime) { }
public void ApplyDispatchBehavior(
ContractDescription contractDescription,
ServiceEndpoint endpoint,
DispatchRuntime dispatchRuntime)
{
dispatchRuntime.InstanceProvider = this;
}
public void Validate(
ContractDescription contractDescription,
ServiceEndpoint endpoint) { }
}
}
}
Ta-da! The only thing to fill in is the actual instantiation of the service type; this could be handled with a Ninject or other Dependency Injection Framework call or it could be a good old-fashioned factory class which pulls in all the various dependencies and returns a ready-to-rumble Service instance.
When I've used WCF to build Web Services in the past, before using this method of dependency inversion, I've been happy enough using the ASP.Net Cache and Request references but now that I'm working with Dependency-Injected classes I've wanted to extract these dependencies so that the Service class can be tested outside of the ASP.Net environment.
The first thing to approach was that I wanted to pass a generic cache reference from the factory. The current thinking (for .Net 4.0 onwards at least) seems to favour the use of the MemoryCache found in System.Runtime.Caching (this Code Project article is a reasonable introduction). So wrote an implementation of ICache (which is internal to my project) that makes use of the MemoryCache. Job done!
Something else I struggled with initially was getting the IP Address of the client making the request (information that I include in the Request logging). This was easy when accessing the ASP.Net Request directly but proved more challenging without it. As usual, some Googling and experimentation yielded an answer. This method may be added to the ExampleWebServiceProvider from the above example:
/// <summary>
/// This may return an IPv4 or IPv6 format address. It will return null if unable to retrieve
/// this information.
/// </summary>
private string TryToGetClientIpAddress()
{
var currentOperationContext = OperationContext.Current;
if (currentOperationContext == null)
return null;
object nameMessagePropertyRaw;
if (!currentOperationContext.IncomingMessageProperties.TryGetValue(
RemoteEndpointMessageProperty.Name,
out nameMessagePropertyRaw)
)
return null;
var nameMessageProperty = nameMessagePropertyRaw as RemoteEndpointMessageProperty;
if (nameMessageProperty == null)
return null;
var ipAddress = nameMessageProperty.Address;
if (string.IsNullOrWhiteSpace(ipAddress))
return null;
return ipAddress.Trim();
}
These are the two biggest things that struck me when moving over to the new Service instantiation approach, I'm sure there will be similar cases but it's reassuring to know that it seems like in most cases the .net Framework has us covered one way or another, it's just a case of finding a different place to look for the data!
Posted at 16:42
8 December 2011
Continuing on with the proof-of-concept I'm doing at work regarding reimplementing a VBScript Engine with WSC Controls in .Net I've been trying to develop an ASP.Net MVC Controller that will execute in the STA ApartmentState having read this article:
http://msdn.microsoft.com/en-us/magazine/cc163544.aspx.
The upshot is that if components that run in STA are shared by something executing as MTA then only a single thread from the MTA worker can operate on the component at a time. If the caller is running as STA then separate instances will exist such that each request (I'm thinking in terms of ASP.Net MVC requests) gets its own instance, preventing requests getting queued up waiting for each other when accessing the STA components.
ASP.Net WebForms Pages support an "ASPCompat" attribute which will create the request as STA, rather than MTA. The article I linked above demonstrates how to do similar for an asmx web service. And the forum answer here claims to describe how to do the same for ASP.Net MVC: http://forums.asp.net/t/1302406.aspx.
However..
I'm not sure what version of MVC that was for, and if things have changed since then (it's marked August 2008), but when I tried to use it it didn't compile :(
So here's the version I'm using with the MVC 3 / .Net 4.0 project I've got on the go - we need an IRouteHandler implementation which makes use of an STA-inducing Handler. Thus:
public class STAThreadRouteHandler : IRouteHandler
{
public IHttpHandler GetHttpHandler(RequestContext requestContext)
{
if (requestContext == null)
throw new ArgumentNullException("requestContext");
return new STAThreadHttpAsyncHandler(requestContext);
}
}
public class STAThreadHttpAsyncHandler : Page, IHttpAsyncHandler, IRequiresSessionState
{
private RequestContext _requestContext;
public STAThreadHttpAsyncHandler(RequestContext requestContext)
{
if (requestContext == null)
throw new ArgumentNullException("requestContext");
_requestContext = requestContext;
}
public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
{
return this.AspCompatBeginProcessRequest(context, cb, extraData);
}
protected override void OnInit(EventArgs e)
{
var controllerName = _requestContext.RouteData.GetRequiredString("controller");
var controllerFactory = ControllerBuilder.Current.GetControllerFactory();
var controller = controllerFactory.CreateController(_requestContext, controllerName);
if (controller == null)
throw new InvalidOperationException("Could not find controller: " + controllerName);
try
{
controller.Execute(_requestContext);
}
finally
{
controllerFactory.ReleaseController(controller);
}
this.Context.ApplicationInstance.CompleteRequest();
}
public void EndProcessRequest(IAsyncResult result)
{
this.AspCompatEndProcessRequest(result);
}
public override void ProcessRequest(HttpContext httpContext)
{
throw new NotSupportedException(
"STAThreadRouteHandler does not support ProcessRequest called (only BeginProcessRequest)"
);
}
}
Then in the routes defined in Global.asx.cs we need something along the lines of:
RouteTable.Routes.Add(new Route(
"{*url}",
new RouteValueDictionary(new { controller = "Default", action = "PageRequest" }),
new STAThreadRouteHandler()
));
in place of
routes.MapRoute(
"Default",
"{*url}",
new { controller = "Default", action = "PageRequest" }
);
This post has been quite derivative of other works but it took me a fair amount of researching to get to this point! Maybe this will benefit someone else going down a similar windy path..
###IRequiresSessionState
Of particular note (and absent from the referenced forum answer) is the IRequiresSessionState implemented by STAThreadRouteHandler. This interface has no methods or properties but identifies the Handler as being one that requires that Session State be passed to it.. er, as the name implies! But without this, the Session property of the specified Controller will always be null. This took me quite a while to track down since - unless you know of this particular interface - it's fairly difficult information to track down! Or maybe I was just having a bad Google day.. :)
Posted at 23:46
Dan is a big geek who likes making stuff with computers! He can be quite outspoken so clearly needs a blog :)
In the last few minutes he seems to have taken to referring to himself in the third person. He's quite enjoying it.