Software system scalability concepts and techniques

When an web application is accessed, the receiving party decodes the image, audio clip or video clip in order to view it or listen to it. This act of encoding and decoding has made the available mechanisms for this process be called codecs -- the combination of cod er- dec oder.

There are well over codecs available for images, audio clips and video clips, with their differences ranging from compression ratios to licensing-royalty schemes.

In web applications though, there's a set of popular codecs used in the majority of cases. Some of these codecs are likely to sound familiar to you, but others may not. Image codecs are the more widely known because they're also used to classify image types. But have you ever heard Vorbis audio clips? Or seen H. You probably have, but as a Quicktime audio clip or Flash video clip, respectively.

The reason audio and video codecs are not as widely known, is due to audio clips and video clips being distributed in container formats e. Quicktime, Flash. For this reason, further discussions in the book on audio clips and video clips is based on codecs. Part II of the book discusses detailed performance considerations for image, audio and video codecs. Character data which consists of the letters, numbers and symbols used in web applications also uses an encoding process similar to images, audio clips and video clips.

Character data is encoded and decoded in several parts of a web application. When you write character data in a text editor or IDE for a web application, it's encoded.

When a user's browser receives a web application's content, it's decoded. When data is saved to a web application's permanent storage system e. RDBMS , it's encoded. When a web application's business logic code reads data from a permanent storage system, it's decoded. This same encoding and decoding process can take place at several other parts of a web application that require reading and writing character data. But unlike images, audio clips and video clips that use the term codecs to refer to the encoding process, character data encoding operates on the basis of character encodings , also called character sets, charsets, character maps or charmaps.

Similar to codecs, there are more than 50 character encodings to choose from. The differences in character encodings are due to different representations of letters, numbers and symbols between countries e. Germany, Japan, France and China use certain character representations in their languages not used in languages in other countries , in addition to manufacturers designing software for a particular country market. In terms of performance, character encodings are important because they influence the space needed to represent characters.

Some character encodings are known as single-byte encodings, while others are multi-byte encodings. Of the multi-byte kind, variable-width encodings are the more versatile, since they can represent characters using 1-byte, 2-bytes or more bytes depending on the character they're trying to represent. Choosing a web application's character encoding solely on the basis of performance i. With variable-width encodings, you can choose a character encoding that's versatile to single or multiple bytes, which is often a better tradeoff -- favoring usability -- than one of strictly following byte efficiency by using single-byte encodings.

To understand this, it's necessary to describe the consequences of using single-byte character encodings in a web application. If you write web applications targeting a particular region in the world, single-byte character encodings like ISO, Windows and even ASCII, can represent the most efficient character encodings.

ASCII which was one of the earliest character encodings, in fact only uses 7 bits out of the possible 8 bits in a byte to represent characters. ASCII served its purpose until computers required to represent more characters. This brought the need to either make use of the 8th bit in ASCII to increment the total number of character representations in a byte to or use multi-byte character encodings to break the limit on single-byte character encodings.

With positions available to represent characters -- double the amount available in ASCII -- many new characters could be supported. Everyone happy, right?

Not exactly, even though characters were enough to accommodate web applications targeting British, French and Spanish citizens, web applications requiring to represent characters in Arab, Hebrew or Nordic languages e. Swedish, Danish, Norwegian , got squeezed out of this character space. This led to multiple ISO regional character encodings.

As well as the equivalent Windows character encoding variations, like Windows supporting Arabic characters and Windows supporting Hebrew characters. This need to encode characters in a single byte leads to usability problems. Care to guess what happens if a web application's permanent storage system uses the ISO character encoding to store data and the business logic code attempts to read it as Windows?

Or a web application's HTML content is written with the ISO character encoding but accessed with a browser incapable of detecting this encoding? You'll see character data as either squiggles e. Yes, you gain performance using a single-byte to represent each character, but this is the potential usability penalty you pay for doing so.

Meanwhile, while this single-byte character encoding fragmentation process took place, users in other parts of the world like China, Japan and Korea found the idea of using a single byte to encode characters laughable. Languages like Chinese, Japanese and Korean rely heavily on ideographs -- symbols representing the meaning of a word, not the sounds of a language -- so with single-byte character encodings limited to representing characters they prove inadequate for these languages that can have thousands of characters.

To address such needs web applications targeting Chinese, Japanese and Korean speakers require to use multi-byte character encodings. By using multiple-bytes, an enormous amount of positions are made available for character representations e.

But would you dare guess what happens if a web application's permanent storage system uses the IECJP character encoding to store data and you use a run-time e. Or a web application's HTML content is written with the GBK character encoding but accessed with a browser in Europe incapable of detecting this encoding? You'll be back to seeing squiggles, question marks or characters with different meaning.

This is because, multi-byte character encodings also define meanings for each position in their bytes e. So is there a solution to all this character encoding madness?

Yes, it's called Unicode , and it's one of the leading variable-width encodings used in software. There are actually several types of Unicode, but for web applications UTF-8 is the dominant choice. The differences consist of UTF-8 using one to four 8-bit bytes to represent characters, UTF one or two bit code units to represent characters and UTF using a single bit code unit to represent characters.

Getting back to UTF-8 which is the preferred Unicode choice for web applications. How is it that UTF-8 solves the problem of multiple character encodings not getting in the way of each other? The answer is simple, in UTF-8 there's only a single character for every byte position that is equally interpreted on every system in the world supporting UTF UTF-8 also assigns each of these characters an exclusive byte position to unequivocally represent such characters on any system in the world.

UTF-8 can do this because it's designed to use up to 4-bytes to represent characters. UTF-8 defines these exclusive byte positions for characters used in languages with Latin letters e. UTF-8 also defines exclusive byte positions for special characters like currency symbols, emoticons and even music symbols, also to name a few. In essence, UTF-8 provides software representations for practically every character imaginable that's used in computers. You can take a look at the entire set of UTF-8 characters by consulting references like the Unicode charts or this Unicode character chart with named and HTML entities.

The first issue than can come to mind about supporting this amount of characters in UTF-8 is the average number of bytes needed to represent characters. If encoding a basic character like the letter A requires using 4-bytes, this can translate into exponential byte growth requiring more bandwidth, memory and storage for each character, all this overhead just to support characters in Ethiopic, Cherokee or Mongolian a web application will never use.

You can relax on this particular issue. UTF-8 is a variable-width encoding character set, which means even though it can use up to 4-bytes to represent characters, it doesn't mean it requires using all 4-bytes to do so, certain characters are represented using just 1-byte, which makes the overhead concerns of exponential byte growth unwarranted.

UTF-8 designers took a very clever approach to the way characters are assigned among this possible 4-byte representation. Can you guess which characters got the privilege of being represented as a single-byte? The most common set used in computers of course, the same sub character set defined in ASCII in the 's. What this means is that the most prevalent set of characters used in software, even encoded using UTF-8, need just 1 byte per character representation, just like single-byte encoding characters sets like ASCII, ISO and Windows So why doesn't UTF-8 use all 8-bits in the 1st byte to represent characters?

Since UTF-8 is a variable-width encoding character set, it needs a way to indicate if a character is made up of a single-byte or multiple-bytes. Therefore one bit of the 1st byte in UTF-8 character representations is reserved for this purpose. Here again, the reason UTF-8 doesn't use the entire 8-bit spectrum across the 4 available bytes, is due to UTF-8 reserving bits in each byte to determine if it's a single-byte code point, a multi-byte code point or a continuation of a multi-byte code point.

But wait, does this mean UTF-8 requires 2-bytes to represent characters that in character encodings like ISO and Windows could be represented with 1-byte? Yes, since one bit in the 1st byte of UTF-8 characters is reserved. But don't get hung-up on small details. How many web applications have you written made up entirely of characters in the upper boundary -- above ASCII characters -- of single-byte character sets?

Not many I would think. Since such characters are used sparingly, the tradeoff between using 2-bytes instead of 1-byte for such characters, is well worth it when you consider the usability benefits of UTF Yes, but again this shouldn't be an issue considering the usability benefits of UTF Web applications that require CJK characters one way or another need multi-byte character encodings, so it's not like UTF-8 adds extra overhead for representing characters that would still need multiple-bytes to be represented.

Even though the maximum character definitions permitted in UTF-8 is 2,,, the most recent Unicode standard version 6. Among these , characters you'll find definitions for the barrage of characters already described earlier like Cherokee and Mongolian, all the way up to Vedic Sanskrit. As you've now learned, choosing a character encoding can impact several parts of a web application, from the configuration and installation of a permanent storage system, to the creation of static content and business logic development.

Throughout the book and where pertinent you'll see several notes indicating the importance of character encoding selection, even though I would recommend you consider UTF-8 character encoding for all your web application needs, given its versatiltiy. The subject of sessions will be a recurring theme throughout the book, one that you will realize is often the crux of most performance and scalability issues.

To explain sessions, it's best to describe their need and how web applications creates them. The first time a user visits a web application, the underlying web framework creates a session on a user's behalf. A session that will hold personalized information needed to customize a web application to the needs of a user, during the time his session is active. Since a session's information is held on the server, all the following requests made by a user will need to have an identifier so the web application can detect which session to associate with each request.

This is due to the stateless HTTP protocol on which the interaction between user i. If a user didn't provide an identifier of this kind on subsequent requests, a user would always appear anonymous.

Practically all modern web framework relies on one of two forms to provide a user with a session identifier: A cookie or a URL parameter appended to all user requests, the latter of which is common when a user has disabled the use of cookies and a technique called URL rewriting. Figure illustrates the various scenarios for assigning a session identifier to a user. As you can see in figure , the first step represents a user making a request to a web application's main page -- index -- illustrating the structure of a request's HTTP headers.

The second step consists of the server responding with the requested web application content. Generally, web frameworks include this last header automatically, providing a unique value for each user. It's also worth mentioning, most web frameworks offer some type of function API for customizing and setting this header. In addition, a cookie's value can also have more data, including an expiration date or be set for specific URLs of a site, based on criteria either set by the web framework or your own requirements.

However, I will describe the use of cookies in detail in part II of the book. When the browser receives the index page, it will inspect the HTTP headers and notice the Set-Cookie header, at which point the browser attempts to save the cookie locally, associating it with the site that sent it.

Next, one of three things can happen. In this case, a request for the reports page would contain this last header. The second case -- illustrated as case b in figure -- represents a user's browser rejecting the saving of cookies.

In this case, subsequent requests made to a site's web application would not include the HTTP header Cookie. When such a request arrives for the reports page, the web framework will be incapable of extracting a session identifier, at which time personalization of the reports page will fail. This second case illustrates the most common response to users denying a session identifier in the form of a cookie, telling them to activate cookie reception to go ahead.

Some web frameworks though, support an alternative to tracking a session identifier when users i. In such cases, when a web framework receives a request for a page like reports that requires a session identifier to personalize, it will resort to URL rewriting.

URL rewriting if often a fall-back option when a web framework detects that the default cookie -- sent on the first response made by an application -- is not being sent on subsequent request HTTP headers. So for example, if a user requests the reports page without a cookie, a user would be re-directed to the originating document -- in this case index -- but with its various URLs rewritten and appended with a session identifier.

This would result in a user requesting a URL in the form reports? Though URL rewriting solves the problem of cookie blocking, its use if often discouraged for various reasons, especially when the alternative of telling a user to activate cookies is relatively easy and less resource intensive.

This is both a performance and management problem. If you have URLs on a page, rewriting them for each user will be a performance hit and a process that can easily be avoided by forcing a user to activate cookies. From a management perspective, you need to ensure the usage of a web framework's HTML markup syntax e.

Another drawback that is often cited for URL rewriting with a session identifier is search engine indexing and bookmarking. This results in an application page never having a 'well known' address.

If you attempt to bookmark the content for a page like reports? Now that you know how web applications track sessions, I will elaborate on the data that is often tracked in sessions. Session data is kept in one of three places: a cookie, a user's session managed by the web framework or a permanent storage system. The easiest way to do this at first is by appending 'John Smith' to the value of a cookie. You could still append this information to a user's cookie, but you would be pushing the limits and purpose of a cookie.

The limit on cookies is bytes or 4 KiB of information, mainly because each request made by a user's browser to a web application sends this information. In addition, it's considered a bad security practice to place personalized information on a cookie e. So with no more reliance on cookies, what do you next? You are left with a user's session managed by the web framework. Since a web framework receives a session identifier in the form of a cookie on each request, a web framework can create a session space to save any type of data required on subsequent requests.

This means that information like 'John Smith' and 'Main Street , New York, New York' can be saved into a user's session and can later be retrieved from any other part of an application to personalize content.

There is literally no limit about what you can hold in a user's session -- held by the web framework on the server -- to build custom content.

However, bear in mind that John Smith is only one user. What if 1, users had an open session with similar information? And this included not only name and address, but dozens of items like telephone, favorite foods and favorite books? You would need a lot of resources. The problem is that each item kept in a user's session takes up resources in the form of memory. So even though 1, sessions with a user's name apparently take up little resources, when compounded with other fields like addresses or favorite books, the amount of memory needed to support 1, sessions can grow exponentially.

At this juncture you will need to consider a storage system for various reasons: a server crash, in which case a user's session data would be lost since it was in memory; wanting to save this information permanently, for a user's future sessions; or offloading session data to permanent storage to reduce resource usage.

Realizing it's not convenient to store a lot of data on a user's session and that it's actually rare you will need every single piece of data to be in user's session at any given time, lets analyze the extreme case of moving all session data to a storage system. You would only leave something like an id in a user's session and rely on this id to access all data from a permanent storage when it's needed.

Though minimizing the data maintained on a user's session is apparently a great idea to cut resource consumption, taken to the extreme it generates another problem with the storage system. Moving session data to a storage system consists of two operations, one is writing it to the storage system and the other is reading it from the storage system when its needed. Format it and find it. Since you've restricted a user's session data -- the one managed by the web framework -- to only an id, lets assume the same 1, users access the application.

Now the application needs to do an initial 1, write operations for each field a user submits, as well as consume CPU cycles to synthesize the data in the format required by the storage system. In addition, each time a user needs personalized content, a web application also needs to read this data from the storage system -- potentially leading to a 1, read operations per field and CPU cycles needed by the storage system to find such data.

As you will now realize, even though sessions are essential to the lifeblood of an application, a careful balance on storing session data at any given moment is essential to meeting sustainable performance and scalability results. The costs of performance and scalability go up as you add more personalization features to a web application.

Since personalization requires knowing a user, this leads to managing a user's session data in either cookies, a web framework's in-memory facility or permanent storage. Throughout the book, you will learn various approaches to managing sessions and reducing the burden they impose on an application's performance and scalability. Communication in a system takes place in one of two forms: synchronous or asynchronous. The behavior of these two communication forms also makes them be called blocking and non-blocking, respectively.

Figure illustrates the process of synchronous and asynchronous communication. As you can see in figure , in synchronous communication a requesting party blocks itself from doing any other work until it has received a response for its request.

In asynchronous communication, a requesting party can continue doing work without receiving a response for its request. As a result asynchronous communication allows a requesting party to do more work. In turn, increasing an application's throughput.

In topics related to web applications, asynchronous communication presents itself in many areas. These areas include web servers reading data asynchronously from hard drives, web browsers making data requests asynchronously a. Not surprisingly, asynchronous communication is often associated with the topic of threads, since threads allow an application to do tasks simultaneously.

With one thread continuing to work while the other waits for a response, this technically means a process is asynchronous since work isn't stopped. However, asynchronous communication is broader and doesn't necessarily need threads. AJAX designs and messaging middle-ware software e. As you will learn in various parts of the book, asynchronous communication can take place in many places that are not directly associated with the use of threads.

In addition, you will also learn how asynchronous communication can enhance an application's performance and scalability. More recently though, software based load-balancers capable of being installed on ordinary hardware or nodes have matured and become quite popular. These software based load-balancers are categorized as reverse proxies. Since reverse proxies are used to apply a variety of performance and scalability techniques in addition to load-balancing, they are further discussed in part II of the book dedicated to static content tier performance and scalability techniques.

Irrespective of form, load-balancers offer more sophisticated strategies for determining onto which node to route requests. These strategies can include determining average loads per node, polling web servers for health-checks and automatically discarding and notifying administrators of unavailable nodes. Due to costs and installation requirements, hardware based load-balancers are mostly reserved for very high-end systems, where as software based load-balancers -- available through reverse proxies -- are used in budget constrained projects, even though these last load-balancers are more prone to configuration errors than their hardware based counterparts, similar to other hardware vs.

Hardware RAID vs. Software RAID. Some cloud computing providers have made it nearly pain-free to scale an application's static content tier. Many now allow you to upload static resources and not have to ever worry about underlying issues like routing, load-balancing, unavailable nodes or inclusively a web server suddenly 'dying' with a sudden influx of traffic.

The only downside to using some of these providers is that you won't be able to customize the underlying software. If you've performed tests confirming your static resources show optimal performance and scalability by using 'x file-system' and 'y web server' you aren't likely to have the ability to install them on a cloud computing service of this type.

The performance and scalability are 'built-in' into these service providers. Another potential downside may be cost, given certain volumes. The next chapter will describe some of the service providers.

You've already exhausted applying performance and scalability techniques, as well as vertical scaling on your application's business logic tier.

Your next option is to horizontally scale it. Lets break down the resources typically associated with a node supporting an application's business logic tier to illustrate the complexities of horizontally scaling it:. Given the nature of an application's business logic tier resources, you will face a sub-set of the same issues presented in an application's static content tier, as well as a newer set given the statefull nature of some of these resources. The sub-set of issues you're already familiar with are those related to establishing a routing policy.

Therefore it will be necessary to carry out a similar strategy, such as round-robin DNS, dynamic name serving or load-balancers. However, the similarities between horizontally scaling both tiers ends there. Figure illustrated this sensitivity. User "John Smith" can initially be routed to node 3, where he can interact with the business logic of an application, but if in this interaction the node generates session data for the user and the user is later routed to node 2, then you can have a serious problem if node 2 knows nothing about the user's session data.

Figure illustrates how to solve this problem. Either all of "John Smith"'s requests are routed to node 3 a. This may sound like a trivial answer, but there is in fact a relatively easy way to do this: decouple an application's business logic tier into parts. Figure illustrates how this 'divide and conquer' approach works. In a monolithic business tier the relationship between business logic is opaque, meaning the logic gets fulfilled through deep code relationships e.

APIs, libraries. When applying horizontal scaling under these circumstances, you're confronted with the situation in figure of having multiple nodes leading to different request pipelines. By decoupling a business logic tier you gain twofold.

A greater leeway to further apply vertical scaling, since now each decoupled part has an entire node's resources to itself i. This decoupling process is also often called a shared nothing architecture , given that each part doesn't share anything with other parts. For a moment you might wonder what implications do the multiple pipelines in a decoupled business logic tier have?

In a decoupled business logic tier, one pipeline fulfills a particular piece of business logic e. Sales , another pipeline another piece of business logic e.

Top , and so on. So what happens if once decoupled, one of the business logic parts e. Catalog can no longer be vertically scaled and requires horizontal scaling itself?

Well you decouple it again , into more sub-systems e. This architecture of having multiple pipelines to fulfill a business logic tier's duties is often called a service orientated architecture. Each decoupled part represents a service, that is then aggregated with other services to fulfill an application's business logic. Such services are designed to interact with one another or inclusively provide part of an application's graphical user interface. The primary requirement for achieving a service orientated architecture is to structure an application's business logic tier with clear and interoperable interfaces, contrary to opaque techniques by means of APIs or libraries.

Many techniques have emerged to achieve this, among the primary ones you'll find the following. Web services are the most recent approach used to decouple business logic. Both techniques consist of exchanging information, generally of the XML-kind or similar markup-variation e.

By using such a format, interoperability between sub-systems becomes easier to achieve, given XML's wide support across several programming languages and platforms. What SOAP and REST do is define the way these data exchanges take place, from supported operations, discovery mechanisms, interface definitions to error handling.

SOAP is the more elaborate approach, offering integrated features like security, transactions and guaranteed delivery. This makes SOAP better suited for decoupling mission-critical business logic e. However, this same set of features makes SOAP substantially more difficult to implement, requiring more forethought to create both the client part of an application, as well as the corresponding server-side part of an application. This fact makes REST far easier to implement and understand.

It works just as when you request web pages or submit information through a web form, information is exchanged between client and server using HTTP methods e. The REST approach is widely used not only by web applications designers in general, but also by many of the major Internet portals, since it offers an easy way to exchange data between sub-systems. HTTP isn't designed to deal with scenarios involving advanced security, transactions, failures or guaranteed delivery, though you could shoe-horn part of an application's code to deal with these issues, you would just be re-inventing what the SOAP protocol is already designed to do.

Therefore, REST is best suited for communication between sub-systems that have little to no side effects. If REST is used to retrieve a sales catalog or a user's profile, a failed or duplicate request is of little consequence. If REST is used to update a sales catalog or create a user's profile, though doable, you have to be more careful in how you approach its design, since duplicate requests can have adverse consequences.

If REST is used to make financial transactions, the underpinnings of REST make it an extremely risky choice, given the provisions that need to be taken into account in this scenario e.

AJAX is a technique that allows web application clients i. AJAX is beneficial for reducing web application latency, since it allows browsers to load an application's data asynchronously i. Messaging technology was one of the earliest approaches to decoupling an application's business logic.

Though there are multiple messaging technologies from which to choose, in principle all messaging technology works alike. One system sends a message to an intermediate sub-system, where its later taken by another or several sub-systems.

This intermediate sub-system is why messaging systems are often cataloged as middle-ware. Figure illustrates a messaging technology architecture. The architecture imposed by messaging technology has a lot of traits related to resolving performance and scalability issues. I've already stated the obvious which is decoupling an application's parts. But in addition, messaging technology also allows asynchronous communication, which in turn allows systems to run in a non-blocking fashion which contributes to greater throughput.

Another characteristic often associated with messaging technology is fault-tolerance. Finally, another characteristic of messaging technology is its interoperability ability. The same brokering of messages between sub-systems, serves as an ideal medium to bridge any interoperability issues -- just like XML in web services. A system based on Java or.

NET can emit messages, that can later be consumed by systems written in Python or Ruby. The only thing that's required is that the systems involved agree on the messaging technology.

Besides web services and messaging technology, there are several other technologies used to decouple systems. One technology that enjoyed great success prior to the emergence of web services was CORBA, which in fact is still widely used in some corporations.

Similar to web services in allowing interoperability irrespective of a system's implementation language, CORBA also provides a language neutral manner for sub-systems to communicate with one another. However, as robust a technology as CORBA is for certain applications, it can become extremely complex to implement. Requiring the deployment of intermediate brokers called ORB's on both client and server, working with an intermediate language called IDL to create both client and server, as well as some advanced skills to configure and put everything together.

With the emergence of SOAP providing a similar set of mission-critical features, as well as a similar language neutral approach, CORBA has lost some of its appeal as a decoupling technology. Moving down the line, we come to language specific ways to decouple an application's business logic.

These approaches are less favorable than the ones I mentioned previously because they are pegged to a particular programming language or platform. Unlike web services, messaging systems or CORBA, which can effectively bridge platform differences, these other RPC technologies require that both client and server be aligned with the same technology.

Among these technologies, you can find. So you've come to the point of having a monolithic business tier, that for whatever reason e.

This is a tough situation to be in, because you're going to have to apply one of the solutions presented in figure , which will inevitably require a custom software layer either written by your team or offered by some third party. As described at the start of the book, creating a distributed computing environment is fraught with pitfalls that are often not clear.

So while it may seem simple at first sight, there is a high-probability you will overlook certain aspects and fall into one or many of the fallacies of distributed computing. The only reliable solution I've seen for solving "server affinity" on an application's business logic tier is through a load-balancer. The reason is because a load-balancer is placed in front -- as the initial entry-point -- of an application's business logic tier.

In this case, a load-balancer works by inspecting requests and detecting information like Cookies or Sessions, redirecting requests to the same node on the initial request and subsequent requests. In fact, some would say it's simpler than trying to decouple an application's business logic, but I wouldn't agree on this last point, a quicker solution perhaps but not necessarily simpler.

By introducing a load-balancer into a web application's business logic tier, you face two additional issues in a production environment. The overhead of installing, configuring and maintaining a load-balancer -- hardware or software based -- as well as the need to deploy multiple copies of a web application's business logic tier, a process identical to horizontally scaling a web application's static content tier.

In contrast, by decoupling a web application's business logic tier you avoid horizontally scaling the tier altogether, keeping a single copy of a web application's business logic in a production environment. No load-balancer or multiple business logic tier copies to worry about. So be warned, even though introducing a load-balancer with multiple copies of a web application's business logic tier might seem like a quicker solution than decoupling its parts, it can eventually become much more complex to maintain.

So as difficult as decoupling a web application's business tier can seem, in the long run it's perhaps the better route to take prior to embarking on horizontal scaling. This is due to the interactive nature of end-users with a web application's business logic.

One second they can submit data, while the next minute they expect to modify the same data, this requires all participating nodes to replicate and synchronize their data, often in a timespan of seconds.

As already mentioned, this particular replication and synchronization process between nodes fulfilling a web application's business logic tier is best left in the hands of a third-party software, not your own initiatives. Certain third-party software used for achieving this functionality requires that you incorporate it into an application from the outset, through an API. So if you thought decoupling parts of an application's business logic tier was difficult, try re-writing parts of an application just so it's able to replicate and synchronize itself to achieve horizontal scaling.

In this sense though, there has been significant progress by some of third party providers, offering very non-invasive techniques for applying replication and synchronization to standard programming language constructs. But once again, you may fall victim to the programming language or web framework you choose from the outset.

Some programming languages have a dearth of options for incorporating replication and synchronization into an application's business logic tiers, while others have a multitude of options from which to choose.

Once again we come to the inevitable comparison of how certain cloud computing services address the issue of scaling an application's business logic tier. A series of cloud computing providers offer the ability to scale an application's business logic tier, as if it were a single node. Underneath though, these cloud computing providers run on a software layer that allows them to replicate and synchronize business logic data -- just like it was described in the previous section 'But what if you can't decouple'.

In this sense, similar to an application's static content tier, cloud computing allows you to scale in a pain-free way, since you don't have to worry about the particularities of configuring or administering the software layer required for this to work successfully. However, among the downside you may find is that you'll be locked into a provider's 'software layer' which is what can allow you to scale in a pain-free way.

The next chapter will describe some of these service providers. To better understand the complexities of scaling the permanent storage tier, I recommend you re-read the section presented in the last chapter entitled 'Permanent Storage Systems', as the strategies described next are based on the concepts and technologies presented there.

Similar to an application's business logic tier, it's a fundamental premise to have a single and unified permanent storage tier in order to successfully apply horizontal scaling to it. However, even though horizontally scaling a permanent storage tier shares some of the same characteristics of the previous tiers, it's still has a unique set of characteristics that make it different. But before even going down the path of horizontally scaling an application's permanent storage tier, I would suggest the same approach as an application's business logic tier: avoid it at all costs.

Simple right? So do you also decouple an application's permanent storage tier? Not exactly, decoupling presents its own set of problems for this particular tier that I will address shortly. What you can do is limit the demand placed on this tier, which can push you away from horizontally scaling, you do this by implementing caching strategies on an application's business logic tier.

You see, unlike the business logic tier whose increased demand depends largely on the amount of end users an application is attending, you have more control over the only client that places demand on an application's permanent storage tier: the business logic tier.

Even though a business logic tier's clients i. However, in the case of a permanent storage tier's client -- the business logic tier -- you can perfectly know beforehand on which data demand occurs the most and implement aggressive caching strategies to lower the load on a permanent storage tier.

Since this is related to an application's business logic tier, part III of the book will address some of these caching strategies you can use to reduce load on an application's permanent storage tier. Moving along, I'll assume you've exhausted all vertical scaling possibilities, as well as the performance and scalability techniques -- covered in Part IV -- of an application's permanent storage tier. You now have to do horizontal scaling to offset load from this tier.

Well, you're back to a scenario like the one in figure , which node holds a user's data i. Or how do you perform replication and synchronization tasks? Similar to an application's business logic tier, it's left to a permanent storage tier's software layer to enforce these distributed computing tasks.

And like so, there are varying degrees of complexity to configuring and managing this type of setup, depending on the storage system you use.

Some are designed out of the box with these features, while others need much effort to run in this way. What I will do next is describe the various approaches used to horizontally scale an application's permanent storage tier.

And in this same process, also include the permanent storage solutions that rely on said approaches. The easiest way to horizontally scale an application's permanent storage tier is to use out-of-the-box cluster software.

Out-of-the-box cluster software allows an application's permanent storage tier to run across multiple nodes and act as a single and unified tier. Forget about dealing with 'server affinity' or replication and synchronization issues, an out-of-the-box cluster software is designed to take care of all these things internally. To a permanent storage tier's primary client -- the business logic tier -- everything is done through a single pipeline.

Under the hood though, a cluster's software executes all kind of duties to solve the distributed nature of operating multiple nodes. These duties can include enforcing ACID properties, load-balancing, fault-tolerance and other concepts particular to horizontally scaling an application's permanent storage tier that I will explain shortly.

Depending on the permanent storage solution you initially choose, the path to using out-of-the-box cluster software can be very straightforward or complex. Some permanent storage solutions offer a clear path of migrating from a standard version i. These migration processes are particularly common among RDBMS vendors, which offer less-expensive licenses for standard versions running on a single node, with the possibility of upgrading to a cluster version capable of operating across multiple nodes.

On the other hand, relational database gives you a lot of access to information with the tradeoff of row-level or table-level locking issues. Writing thoughtful and clean code is often neglected, but unbeknownst to most, it plays an essential role in scaling your software. A high-quality code input produces longer-lasting output. This means that when a code is well thought of, it will be easier to duplicate and test in the future.

Clean code is simple and readable in the eyes of other programmers. A well-written code makes it easy to implement future changes. With that in mind, this highlights the need for a good developer. You need a programmer that will not only do the job but will do it well. Lastly, you need to ensure that your software is convenient for testing and maintenance. Once you decide to improve it, your software will undergo countless testing and deconstruction.

When you set it up for automated testing across functionality, bugs will efficiently illuminate in your system and will make the process much faster and more manageable. In short, you make it friendly for your QA specialists. Your software development team has to be prepared to take on drastic changes for your startup to grow. Additionally, you need the right resources to help you grow.

What are your first steps? If you need mentors, Matt DeCoursey and Matt Watson are startup heroes that have helped several aspiring entrepreneurs succeed in their business. They are the brains behind the offshore software development company, Full Scale.

Full Scale helps early-stage businesses grow and succeed by giving them access to affordable talents and resources. The company caters to all your development needs, from highly skilled developers of any specialization, graphic designers, content writers, project managers, and successful entrepreneur leaders. Contact us and learn more about our services today!