Two years ago at Allegro we used to have a very typical Big Data technology stack. The architecture was based on a Hadoop cluster and we would query it with plain Hive queries, Spark jobs and Jupyter notebooks. Over those last two years we have transformed it into a more efficient and easy to use OLAP platform.
At Allegro we use Solr as our main search engine. Due to the traffic to our search engine being thousands of requests per second and index size on the order of a hundred million documents we need to develop custom optimizations. In this post I will describe the story of our Solr plugin development.
We messed up. On July 18th, 2018, at noon, Allegro went down and was unavailable for twenty minutes. The direct cause was a special offer in which one hundred Honor 7C phones whose regular price is around PLN 850 (about € 200), were offered at a price of PLN 1 (less than € 1). This attracted more traffic than we anticipated and at the same time triggered a configuration error in the way services are scaled out. This caused the site to go down despite there being plenty of CPUs, RAM, and network capacity available in our data centers.
Lately my colleague Michał described how he tracked a Java memory leak. Although that problem was completely solved, new complications suddenly appeared on the horizon. As it usually happens, everything started with an alert from our monitoring system which woke me up in the middle of the night.
Throughout my studies at university and work in the industry I switched my primary programming language from Java to C# and back again to Java. This article gathers some of my thoughts on using both languages. It’s not intended to be a comprehensive comparison of Java and C#. There are a lot of other resources on the Internet that cover this topic. Instead, I want to focus on what I personally liked about both languages and how it felt to transition between them.
The following article has two parts. The first part describes improving Allegro iOS app launch time by adopting static linking and sums it up with a speedup analysis. The second part describes how I managed to launch a custom macOS app using not-yet-fully-released dyld3 dynamic linker and also completes with an app launch speedup analysis.