Before I write down anything else, I would like to drop in a single learning which I carry out throughout every task and forms critical importance of all the steps

1. Focus on all the small things You, for sure will be able to appreciate this point when you go out in the market. This is something which is going to give you much added advantage over everything else with a comfortable entry barrier. With people to work and care taken while execution of every single step, things always results in beautiful outcomes.
2. Gather lot of related market data for analysis Lets come back to analysis now assuming that above mentioned sort of advice will always be given an utmost importance. First part of every plan starts with market or industry analysis. Even if your idea or plan seems overwhelming enough for you to skip this thing, you should be very strict to yourself gathering negative data for you too to make you strong enough to fight against them when the right time comes. Read lot of related blog (I prefer Google Reader ) as others’ experience is always going to help you a lot. I would rather advice you to make an account over delicious and preserve all your bookmarks there so that you can share with me or even yourself later
3. Not only this… but ….  principle This is why I am writing this blog. As you may have already figured out by now, you are going to be remembered for one “extra” thing which makes you distinguished and will be adding value to your brand. Now it makes sense to give lot of importance to that “extra” thing. Its a very common practice in fact, to focus on “extra” values you providing deviating from your core, e.g., lets assume that you are creating wishlist of books and then doing analytics over wishlist to suggest best deals to users from your supply chain. Your aim should not be just managing wishlist and doing intelligent analysis over that, but the aim should be “Not only providing best deals on books, but also suggesting people to buy books which they like and has good deals available”. Now always remember that “not only” part forms core of your business and you ought to make it sure that you are very strong in this part. “But” part is something from which you are going to be distinguished. Its always a learning process and you ought to make it sure that you are very flexible and adoptive to market behavior for this part.

Would like to conclude by someone’s status I read somewhere

“Build it and they will come”

The idea is to start with a register filled with some arbitrary pattern, then shift it right (say) a step at a time, filling in vacated positions from the left with a bit determined by the contents of the register. The diagram below shows a simple 4-bit register, with the new bit taken as the “exclusive or” of the two rightmost bits.

So, if there are 4 bits b1, b2, b3, b4 and b1-b2-b3-b4 is any random number generated; then the next number which will be generated will be such that b4(new) = b3, b3(new) = b2, b2(new) = b1 and b1(new) = b3XORb4.

The above example is for 4 bits and the algorithm is replicated for n bits too.

The problem in hand is to observe if any random number picked as seed will span all the possible numbers or not. I found a pretty elegant solution for the problem. Taking seed as 0000 clearly shows that cycle for such random number generator will be 1 as next operation on 0000 will lead us to 0000 itself. Lets see what happens when we take 1111 as seed. The cycle is as shown for the taken seed:

1111->0111->0011->0001->1000->0100->0010->1001->1100->0110->1011->0101->1010->1101->1110->1111

Clearly cycle spans complete range except 0000.

Now it becomes obvious that any seed taken can be either 0000 or one number among the cycle mentioned above. If seed taken is 0000, cycle is of size 1 and hence it should be a must check that seed is not 0. Any other seed taken belongs to the above mentioned number generation cycle and hence will have cycle length of 15 (2^n -1 if n instead of 4). So the span is total range except 0. Thus additive congruential method is very strong in terms of spread of values it takes given the seed is not 0. You pick any other random number and you always get next random number as unique number only ofcourse till the cycle is not complete / all numbers are not used exactly once.

The preliminary work that needs to be done is as below :

1. Defining your target market. Target market is business dependent and you need to have a very good idea in mind for what users are  most expected to buy or use your service or product. In target market, one needs to mainly define the target age and income groups.
2. Next step involved is doing the basic keyword analysis. By keyword analysis, we mean that you need to dig out the numbers of relevant keywords being searched over internet revolving around services you providing or things you selling.
3. Do target market viz-a-viz sector analysis. You need to have a good statistic about target market behavior over internet. Let me explain this with a very simple example of e-commerce. Suppose you want to sell things online and you are planning to set up an e-commerce portal over net. Now the behavior study shows that whole selling industry can be broadly divided into three categories. First are content portals – these are the website which publish content majorly for brand awareness of things or services selling over internet or to introduce new stuff to visitors.  Second are review portals - these are the websites where people share their reviews about different services or products and collaborate to pick out the best services or products versus how much they are priced for. Third comes selling platforms – which are very much targeted to sell things only. Now the user behavior is: on an average, every person planning to buy anything over internet visits atleast 2 content portal on the product, some view reviews as well and then finally plan to buy the product. Other behavior learning is that the final buying is done through portals which are only selling stuff. Now, this learning can be used to understand that selling things is not just setting up a selling platform only but also to drive your narratives in review as well as content portals which will be very important.

A logical marketing strategy making logic of every spent can then be prepared. Generally we land up in two situations after the work done mentioned as below:

1. Lot of searches are  being done revolving around product or service you are trying to promote and sell. By this, we say keywords’ search volume for your business is high. In that case, you should focus on tapping search results and presenting a very targeted page in search pages for the keyword searched. These targeted pages are termed as campaign pages and are proved to be the most helpful way to create leads for your business. The creative of the campaign page too matters a lot. An example of target page/campaign page can be seen here – http://travelomate.com/c/luxury/golden-triangle.php. The other way will be to make land your user to the awareness page mentioning why you are different and how you can add value to them/their business where you put down effective content as text, picture or video.
2. Keywords’ search volume for your business is low. Do not worry. The only problem here is there is no online awareness of things you are doing and this would also mean that you probably will be the first person to bring your business on those lines. A very strategic Brand Awareness Campaigning is prepared in which once you have defined your target market, a social network media planning is done targeting your target market as was defined and calculating what percent of target area’s market will be hit by your brand campaigning. The models for the campaigning will be Pay per Click, Pay per Impression and organic marketing along with which is known as Social Media Marketing.

That is the basic overview for the learnings before you start your project or thoughts. Good luck for everything.

The above picture shows the website’s Time spent downloading a page (in milliseconds) which is taken down to 1.23 seconds average from initial 16 seconds.

Following were the optimizations which were done to achieve that:

1. Compressed CSS and JavaScript files to one file
2. Created widgets publishing strategy according to the website traffic.
3. Optimized in page javascript to a lot.
4. Hosted all the images to images.sitename.com in an NGinx server which is known for serving static content faster.
5. Did <a name> mapping to serve files at different places and made the whole architecture independent of each other using various integration API’s
6. On process of doing more

The purpose of the blog is, apart from all other scalabilities, these things are most essential to do and to figure out lot more other things so that page load time comes down.  If you have any site, feel free to ping me and ask for optimization strategies. And if you are targeting lot of Google optimizations, doing page load optimizations is THE first step before onsite content optimization even, the reason being you need to ensure that pages crawled frequency should be high if you are spending on content.

Shit happens while coding and so was done by developers of the module.

Here is the learning part:

The javascript file of the module contained following code:

As this javascript file was loaded, the already loaded windows.onload function to load Google Maps was over ridden. Since, I had to use both the module as well as Google Maps plugin, this is how I altered the code:

So the learning is never use windows.onload event stand alone especially when you are making module or third party service or widget. Always detect onloaded functions and append to them whatever you want to initiate.

Lot of learnings to do still. Happy coding

I will be requiring lot of inputs from readers to correct this architecture discussing various problems with it along with enhancements that can be done. Will really appreciate inputs. I can also be contacted at parag dot arora at gmail to discuss a better architecture for the given need.

hardware-architecture

• Web Servers: This will be first level of geographical mapping of servers. Web Servers will be also having a view caching so that the view loading is enhanced in terms of speed. The scaling of web servers will depend on where (in terms of geography) the traffic has been increasing. A person sitting in US will be actually accessing a server based in US and person sitting in India will be accessing a server based in India. Such mechanism will ensure fast connectivity as accessibility time will be reduced. There will be a master server where the IP’s will be mapped to server address in a hash table which will be cached in RAM and after detecting client’s IP, we will direct the client to the server in their area. To start with, this master server will act as the web server and the scalable web servers mentioned will be introduced as we start entering into other countries. The best mechanism will be to have dedicated domains like .co.uk, .co.in, .co.us etc. for web server mapping.
  
• App Servers: Web server will interact with application servers where the business process will be done. The application servers will host business process of features and will form a bridge between web servers and database. Each web server will be mapped to one or more application servers which will reside in the same geographical location as of web server. Initially, in the first phase, application server and web server will be physically same machines. Most of the AJAX functionalities will be served via these servers only. All the dynamic information will be served via these server and web servers will host only template files and javascripts or more precisely the client side interaction code. Sessions will also be saved in app servers.
  
• DB master servers: Application servers will directly interact with database master servers. Sharding will be done in database which will be done through master-slave architecture using entity lookup services. The core purpose of master servers will be to maintain slave mappings w.r.t. entities. e.g., if there are lots of micro updates entered by users, then we will use sharding techniques to save these updates. Every user will be mapped to a slave database servers containing micro updates table and master server will maintain a memcache which will contain a table mapping users (or a user groups to enhance efficiency and decrease data stored in master servers) to slave servers.
  A problem exists for such an architecture. Suppose the user lives in India and web server in India connect to App Server which try to lookup user in master server memcache. If users try to connect from India, then there will be no problem. Now suppose that the same user goes to Canada and try connecting to our website. Then obviously memcache placed in Canada will not contain this user record.
  The proposed solution for this problem is, we will maintain the universal copies within the database and memcache will only contain the active users’ data. If the user (discussed) record is not found in memcache of master db server in Canada, it will query the user record from the copied database and save it in memcache.
  The slave access will obviously be little slow in this case since the data finally is retrieved from server based in India but this is obvious as we should not maintain lot of copies of same data. This server will require lot of data copying and will be little inefficient in terms of storage but will be useful if the users belonging to website shifts their locations too often (more travelling users).
  The other solution to this problem can be master web server will contain the access server of every user and in addition of keeping track of ip of user from next login, it will also contain user – access server mapping in the lookup table. This approach assumes that most of the users will not shift countries too often.
  
• DB slave servers: Wherever sharding is used, master servers will save, update or retrieve data from these slave servers.

1. Software

As soon as the business problem is clear, following are the approaches which one IMO should follow

1.1 Entities

The first is figuring out entities. The entities should be defined in such a way that objects of similar entities can be formed out of entities.

1.2 Relations

The next step is finding relations among the entities. Relations should be well defined and the entity relationship diagram should be clearly understandable to a layman too.

1.3 Database

Next step is mapping these entities to database tables. The relations should also be defined in the database model and depending on the datamapping classes or approaches you are going to use, you need to make these relations fit into tables or create separate tables for relations (read about database 5^th normalization or 5NF).

2. Hardware

2.1 Needs

For deciding the hardware architecture, first thing is to figure out all the needs. e.g., needs can be, there will be lot of micro updates (like in twitter ) or lot of photos (like in facebook). Then one of the standard approaches, like database sharding using lookup services (for these examples) can be used. It is thus very important to understand the needs of the system in terms of requests as well as transfers and storage before deciding any hardware level architecture.

2.2 Solutions

Once the needs are defined, hardware solutions can be thought of one by one to sort out the problems to be tackled. Initially application can be started with one server architecture but it should be scalable enough to do all sort of sharding, master – slave scaling etc. in just few clicks. The scalability of architecture will be decided by how well the application can be scaled to solve problems in different ways. For that you need to be aware of standard solutions . Be sure that you know about standard approaches like profiling, indexing, caching, monitoring, performance testing, schema normalizations etc before you actually make the final hardware architecture.

2.3 Hardware Architecture

Once you have figured out all the problems from top to bottom and thought about all the solutions to handle these issues, you need to integrate all the solutions to make one scalable architecture diagram of hardware. Also try giving you servers some name which can be used for mapping things while sharding etc. in case in future. It is very important at this point of time to document all the scalability solutions you have thought of.

3. Dev

3.1 Dive Hard

The final step is to dive hard into development. It is always useful to use standard approaches like MVC, ORM for a flexible web design.

Source: http://www.slideshare.net/joanna_maxwell/creative-project-planning-ideas-1801931