• Latest post This Fall digest

    Some of our engineers run their own tech blogs. We encourage them to move here, but for various reasons, they prefer to publish on private blogs. We respect their decisions. What we can do is to gather all the blog posts published by allegro.tech engineers around the web in the one place.

  • Golang slices gotcha

    In this post I present a story of a bug that hit us recently. Everything was caused by unexpected (although documented) behavior of Go built-in function append. This bug has lived silently for nearly a year in allegro/marathon-consul. Ensure you run the latest version.

  • Presto - a small step for DevOps engineer but a big step for BigData analyst

    I bet you have found this article after googling some of the issues you encounter when working with a Hadoop cluster. You probably deal with Hive queries used for exploratory data analysis that are processed way too long. Moreover, you cannot adapt Spark in your organization for every use case because of the fact that writing jobs requires quite strong programming skills. Clogged Yarn queues might be your nightmare and waiting for the launch of the container when you run even a small query drives you mad. Before we deployed Presto — a Fast SQL engine provided by Facebook — our analysts struggled with these problems on a regular basis. Introduction

  • The Agile Testing Days Conference - Potsdam 2016

    Last year Agile Testing Days Conference was held between 6th and 8th December in Potsdam, Germany. According to statistics provided by organizers there were around 600 attendants and about 30 speakers in keynotes / workshops. During the whole event around 22 different workshops took place, 9 keynotes and around 30 other speeches / presentations divided into 7 parallel tracks. I had the opportunity to attend the first two days of the conference and in this article I’d like to share with you a short review of the most interesting sessions and my impressions about the event.

  • Distributed rate limiting of delivery attempts

    In our services ecosystem it’s usually the case that services can handle a limited amount of requests per second. We show how we introduced a new algorithm for our publish-subscribe queue system. The road to production deployment highlights some key distributed systems’ takeaways we’d like to discuss.

  • Hitting the Wall

    Running Mesosphere Marathon is like running… a marathon. When you are preparing for a long distance run, you’ll often hear about Hitting the wall. This effect is described mostly in running and cycling but affects all endurance sports. It happens when your body does not have enough glycogen to produce power and this results in a sudden “power loss” so you can’t run anymore. At Allegro we have experienced a similar thing with Mesosphere Marathon. This is our story on using Marathon in a growing microservice ecosystem, from tens of tasks and a couple applications, to thousands of tasks and over a hundred applications. If there is no mention of Marathon version, it is 1.3.10 and below; we need some time to test and deploy the latest 1.4 release. If you are interested in how our ecosystem is built, take a look at below MesosCon presentation.

  • Estimating the cache efficiency using big data

    Caching is a good and well-known technique used to increase application performance and decrease overall system load. Usually small or medium data sets, which are often read and rarely changed, are considered as a good candidate for caching. In this article we focus on determining optimal cache size based on big data techniques.

allegro.tech

At Allegro, we build and maintain some of the most distributed and scalable applications in Central Europe. This poses many challenges, starting with architecture and design, through the choice of technologies, code quality and performance tuning, and ending with deployment and devops. In this blog, our engineers share their experiences and write about some of the interesting things taking place at the company.