Saturday, December 02, 2017

Stack overflow is built using ASP.NET, C#, SQL Server, Windows, IIS...

Irrespective of what technology we work with how many times have we landed on Stack Overflow / Stack Exchange website on a daily basis? If we ask this to any developer the answer would mostly be MANY times in a day :)

Site was launched in 2008 and is built using Windows, SQL Server, IIS, and ASP.NET along with HAProxy, Redis, and ElasticSearch, all served via Fastly CDN.

Fun Facts:

  • 1.3 Billion page views per month
  • They transfer ~55 TB data per month
  • 4 SQL Servers (organized as 2 clusters)
  • Stack Overflow serves 528 Million queries per day (Peak 11000 queries per second)
  • Stack Exchange, Careers, meta serves 496 million queries per day (Peak 12800 queries per second)
  • Over 360 databases with the same schema, which changes frequently
  • They made use of Microsoft Bizspark program before getting graduated
  • All their production traffic is served using physical servers & 
  • Cloud services are being used only for storing encrypted offsite backups (Glacier) and for DNS (Route53)


Programming stack used:

  • C# + ASP.NET MVC, 
  • Dapper ORM, 
  • StackExchange.Redis, 
  • DotNetOpenAuth, 
  • MiniProfiler, 
  • Jil


Deployment on a typical day:

  • 25 times per day code deployment to the development server (just Stack overflow Q&A alone)
  • 5 to 10 times a day production deployment happens!


Page rendering time:

  • Homepage renders @ 12.2 millisecond
  • Questions page renders @ 18.3 milliseconds


Source: https://goo.gl/z9cfex & https://goo.gl/EYwTZY

Friday, November 24, 2017

What is common between these apps?

What is common between these desktop applications/tools?


  • Slack
  • Wordpress
  • GitHub Atom Editor
  • GitHub desktop
  • Microsoft Visual Studio Code
  • Microsoft SQL Operations Studio


All are built using Electron Framework (Open Source)


  • Electron framework lets us write cross-platform desktop applications using JavaScript, HTML, and CSS. It is based on Node.js and Chromium. 
  • It allows us to (re)use web components - HTML, CSS and JavaScript in the creation of desktop applications. 
  • Then it can be distributed across platforms - Windows, Mac OS X, or Linux.

Saturday, November 11, 2017

Ten Quotes From Satya Nadella's Book "Hit Refresh" That Inspired Me

A few years back Microsoft offered a free upgrade to Windows 10. That was an interesting move as their main competitor Apple was doing it for a long time now. As a consumer felt that Satya Nadella becoming CEO of Microsoft has brought in quite a lot of changes and they started embracing open source which initially was surprising to me.

So for me, it was a no-brainer to pick his book "Hit Refresh" from the store immediately once it was released. The first half of the book was more interesting where he was talking about his life in India, about his kids, love for Cricket and how over a period he got more interested in Computers than Cricket.

Microsoft CEO Satya Nadella's Book


One anecdote of how he was first interviewed at Microsoft was exciting. He was asked
- "Imagine you see a baby laying in the street, and the baby is crying. What do you do?".
- As a young engineer, he has answered the question exactly as I would have: "You call 911".
- The interviewer's response was cooler: "You need some empathy, man. If a baby is laying on the street crying, pick up the baby.".

By the way, these are the 10 quotes from this book which I found to be Inspiring. 

  • The C in CEO stands for Culture
  • Success can cause people to unlearn the habits that made them successful in the first place.
  • Learning to fly is not pretty but flying is.  
  • It is not the device that is mobile. It is you.
  • Culture eats strategy for breakfast.
  • We needed to build deeper empathy for our customers and their unarticulated and unmet needs. It was time to hit refresh.
  • Empathy grounds me and centers me
  • A leaders job is to "Find rose petals in a field of shit"
  • Leadership means making choices and rallying the team around these choices.
  • For anything great to happen – great software, innovative hardware, or even a sustainable institution – there needs to be one great mind or a set of agreeing minds.

I loved his thoughts on how the world is not about Man vs Machine but their coexistence, to make this world a better place. In my opinion, overall, "Hit Refresh" is a great read and after reading the book, I'm convinced that Microsoft picked the right CEO at the right time :)

Wednesday, June 21, 2017

Make it WORK then Make it BETTER



Mostly this mantra is followed by majority of the development teams (in particular startups) & it certainly makes sense as long as it gets understood completely.

In my nearly 2 decades of experience, I have seen teams strive tirelessly to "Make it work" and deliver a minimum viable product but are indirectly forced to skip the "Make it better" part due to lack of time/bad estimation.

Why? 

Because once the minimal viable product is made people start pulling resources out of that, start assigning next module so that they can "Make it work". The assumption was we can work on refactoring the code tomorrow or day after which in reality never happens. Ultimately the development folks don't find time to "refactor" their initial code to "Make it better".

"The problem with waiting until tomorrow is that when it finally arrives, it is called today." - Jim Rohn

So what? It is already working anyway! 

"Make it work" which is not followed by refactoring sessions to "Make it better" will undoubtedly break down when we start to scale. Over a period of time code base would become complicated & the cost of change at that time would be too high which might even jeopardize the business altogether.

Make it work 
- If need be, do research and implement a solution to achieve the desired behavior/result.
- The solution might not be having a good design,
- Code will not be optimized and
- It won't be easily maintainable.

Make it better
- Now that desired result is in hand, refactor the code.
- Organize the code base properly to have a good design and
- Make sure it's maintainable.
- Optimize the code to achieve the desired performance goals of the application.


Thursday, January 26, 2017

Is Banning Coke/Pepsi the solution?

Disclaimers:

1. I support Jallikattu and had even posted my thoughts about it here - "Jallikattu needs your support"
2. It's been almost 3 years since I stopped consuming any aerated drinks (Coke, Pepsi, Fanta, ThumsUp, Limca, Sprite etc.,). So I always request people to stop drinking any aerated drinks and instead drink Tender coconut, buttermilk, Fresh juices (without ice or sugar :) ) etc.,

What is the reasoning for asking a BAN?

From what I understand, the reason as to why protesters are asking for a ban is because of the depletion of Thamirabarani river. The depletion that is caused by these companies affect farmers and other people in the surrounding locality. Drought kind of situation is staring us. So the root cause is "water scarcity".

My 2 cents:

Definitely, we got to support our farmers who are the backbone of our country and if something is affecting them then everyone should do our bit to clear the hurdle for them.

While thinking about it the questions which came to my mind are:

-  "Are these companies the only reason for water scarcity?"

- "Who is responsible for ALL encroachment on all our water bodies in Tamil Nadu?" - Shouldn't we Ban encroachments & restore those water bodies?

- "Why selectively Ban Pepsi & Coke alone? Why not other companies which manufacture aerated drinks?" - If the product is the issue, shouldn't one avoid all aerated drinks? As we all know Bovonto is a local Tamil Nadu based company which is there since 1916 and have avoided numerous attempts by other giants to take over them & survived. But at the end of the day, they too manufacture aerated drink only? And they would as well be sucking water from somewhere so won't it affect that locality?

- "What do we do with bottled mineral water like Aquafina (Pepsi), Kinley (Coke)?"

- "After encroachments, the other main culprits are Sand Mining but why isn't anyone serious about stopping it?" - This is actually changing course of river flow, groundwater collapses, rivers gets dried up etc.,

- "How much water is needed to manufacture 1 liter of bottled mineral water, soda, wine, beer & Hard alcohol?". In this October 2013 article (which is based on North American companies) I found these data points:
* 1 liter of bottled mineral water on an average needs 1.39 liter of water
* 1 liter of soda on an average requires 2.02 liters of water
* 1 liter of beer on an average needs 4 liters of water
* 1 liter of wine on an average needs 4.74 liters of water
* 1 liter of hard alcohol on an average requires 34.55 liters of water

I couldn't find any similar data points for Indian companies but I believe it gives us an idea of how much water is getting used to manufacture those products kept on the shelf of our Tasmac!

Though the "Water" problem is for real the proposed solution of selective banning doesn't seem to be a complete solution. To me, it sounds more like an emotional decision than a decision based on facts.

Newton's third law is: For every action, there is an equal and opposite reaction.

By saying we are banning selected companies we would be sending a wrong signal. I am not very sure whether Tamil Nadu can survive without foreign investments and outsourced job in the present globalized environment?

- Aren't we fine with American outsourced jobs in India?
- Aren't we fine with our kids, relatives going for any of the university in the US for higher studies?
- As a reaction, if these companies lobby in the US and try to stop the import of Indian goods there what would we do?

So banning something is not a permanent solution. If those companies have flouted any rules then penalize them based on that. Also if we have proof stating it has pesticides and it's not safer for human consumption maybe take it up with Food safety & standards authority of India (or whoever is supposed to regulate it).

Possible solutions:

1. We have regulations in our system but are those getting really implemented properly? Got to monitor that. So the best place to start with would be to have better regulation and make sure the regulations are implemented properly.
2. Instead of banning we should work towards creating a better product than these and let them lose their market share.
3. Create awareness about the health hazards and educate our friends, relatives to stop using their products. When there is no market for them they themselves will move away.

Reference material for further reading:

1. Article about Sand Mining in Tamil Nadu
2. Quora discussion - Is Bovonto healthier than Pepsi/Coke?
3. Sand Mining issue - Impact cannot even be calculated
4. How much water is needed to make manufacture 1 liter of mineral water, wine, beer, hard alcohol?

Tuesday, January 10, 2017

Jallikattu needs your support

Myth: Jallikattu is a bullfight

Jallikattu is NOT a bull vs human fight. It's a game where the players are required to embrace the running bulls by hanging on their hump as far as possible.

It's NOT a bullfight and the bulls are not injured (or) killed later on. This is in sharp contrast to the Spanish bull fight where the players insert a sword into the spine of the bull!

So is Jallikattu cruelty-free? 

Maybe not. But lets take a step back and think through this.

There have been on field deaths in Football, Cricket, Boxing etc., So did authorities try to make sure the game is, even more, safer (OR) did they ban those sports?

Yes if there are rules there are going to be rule breakers as well. So should we discipline rule breakers (or) ban the entire sport just by looking into few exceptions happened in the past?

There are quite a few rules to be followed in Jallikattu as well. Just in case it isn't followed should the protest needs to be for implementing the rules (or) banning the sport? 

Doesn't it clearly say there are some misplaced priorities here?

When important stakeholders (Farmers, people who love & own bulls, researchers) time & again say

1. Ban on Jalikattu is actually against native breeds,
2. Lots of native breeds have already become extinct & this ban on Jallikattu will probably lead to the remaining 5 native breeds being extinct too,
3. There is a bigger politics behind this move etc.,

Did Peta India supporters / Media take any effort to research on that angle and provide their findings? (or) where they even ready for a healthy debate with those stakeholders on that topic?

Having a different point of view is never an issue but not willing to research, learn, debate and then be ready to change stance based on facts is what is worrisome.

Please don't blindly believe the media & NGO behind this move. We need to save our native breeds. Let us not give away our treasures to groups who are masters of playing "Divide & Rule" game.

Now if our great-grandparents are alive they wouldn't allow this to happen. Why? Because they know the effort, pain, sacrifice they all had to make to get the freedom for India from a "divide & rule" group earlier. Even before our great-grandparents realized the big picture that group had spread their wings all around the country and colonized India.

Jallikattu is the pride of TamilNadu, Part of Tamil Culture, Identity of Tamilians. Let's do our bit to save it from corporates with the vested interest.

For further reading:

1. Jallikattu, The Pinnacle of Tamil Culture
2. Rules of the Game - Jallikattu
3. Banning Jallikattu will decimate India’s indigenous cattle breeds
4. [Article in Tamil Language] - PETA is responsible for destruction of native breeds specifically bulls
5. [Tamil] Speech by Mr. Karthikeya Sivasenapathy who is the founder of Senaapathy Kangayam Cattle Research Foundation
6. India is the world’s largest producer of milk. But in 10 years, we will be forced to start importing it. And the Indian cow will no longer exist.

Monday, January 09, 2017

SQL Server Always Encrpted - At a high level how does it works?

One of the excellent feature introduced in SQL Server 2016 is "Always Encrypted". This gives an extra layer of protection as no one (including the production DBA's) will be able to access the actual data without having the appropriate key.

An high level overview of how SQL Server 2016 Always Encrypted work:

1. Always Encrypted is a client side encryption technology in which a SQL Server client drivers (In our case, it would be ADO.NET) plays the key role. The driver encrypts the data which application sends in its queries to the database, and it then sends encrypted data to SQL Server.

2. Now when the application retrieves the encrypted data from the database the DRIVER transparently decryptes returning plaintext to the client app. Consequently, SQL Server never sees a sensitive information in plaintext. The keys, in fact, are managed entirely on the client side & the server doesn't have access to the keys either.

3. The key can be stored either in a Windows Certificate (or) Azure key vault,

4. Despite the fact that SQL Server never has access to plaintext and sensitive information, or the corresponding encryption key. SQL Server can query the data and can perform certain computations on encrypted data, namely equality comparison, equality joins, exact match searches or group by operations. - Conditions apply :)

5. There are 2 types of "Encryption Type": Randomized encryption and Deterministic encryption

a. Randomized encryption: This algorithm produces a different cyphertext value for a given plaintext value. Therefore, randomized encryption is more secure but it prevents any operations on encrypted data. Only we can select the column data and display that's it.

b. Deterministic encryption always produces the same cyphertext value for the given plaintext value. Therefore, it enables equality comparison on encrypted data in operations such as exact match searches, equality joins or group by operations.

c. Security Concern?

There is a slight security concern related to deterministic encryption because each plaintext value always maps to the same cyphertext value. An attacker can potentially examine the cyphertext patterns and guess the underlying plaintext values - especially if the dataset is small. For ex, columns contain gender information about male and female. So that's something that we need to keep in mind and be careful while choosing the encryption type.

6. How exactly does the encryption happen?

a. It would download the data of the particular table, encrypts it and then uploads it back. In that process, the schema of the table would change.

b. Encrypted with clause has 3 parameters. The name of the column encryption key protecting data and the column, the type of encryption (deterministic / Randomized) and the name of the encryption algorithm.

c. Always encrypted currently supports just one encryption algorithm, which is AES256.

7. Sample table structure post enabling "Always Encrypted" feature:



8. Web.config changes:

This is the key part because we will see that actually to enable always encrypted add the below value in your connection string.

column encryption setting=enabled;

9. Now application would be able to decrypt information retrieved from the database because it has access to that certificate, that we produce in the wizard. It's basically running on the same machine where the certificate was generated and stored in windows certificate store. And then it is able to decrypt a column encryption key and subsequently, decipher the sensitive information and show it in the clear text as expected.