• Latest post Mesos Executor

    Apache Mesos is an open-source project to manage computer clusters. In this article we present one of its components called Executor and more specifically the Custom Executor. We also tell you why you should consider writing your own executor by giving examples of features that you can benefit from by taking more control over how tasks are executed. Our executor implementation is available at github.com/allegro/mesos-executor

  • This Fall digest

    Some of our engineers run their own tech blogs. We encourage them to move here, but for various reasons, they prefer to publish on private blogs. We respect their decisions. What we can do is to gather all the blog posts published by allegro.tech engineers around the web in the one place.

  • Golang slices gotcha

    In this post I present a story of a bug that hit us recently. Everything was caused by unexpected (although documented) behavior of Go built-in function append. This bug has lived silently for nearly a year in allegro/marathon-consul. Ensure you run the latest version.

  • Presto - a small step for DevOps engineer but a big step for BigData analyst

    I bet you have found this article after googling some of the issues you encounter when working with a Hadoop cluster. You probably deal with Hive queries used for exploratory data analysis that are processed way too long. Moreover, you cannot adapt Spark in your organization for every use case because of the fact that writing jobs requires quite strong programming skills. Clogged Yarn queues might be your nightmare and waiting for the launch of the container when you run even a small query drives you mad. Before we deployed Presto — a Fast SQL engine provided by Facebook — our analysts struggled with these problems on a regular basis. Introduction

  • The Agile Testing Days Conference - Potsdam 2016

    Last year Agile Testing Days Conference was held between 6th and 8th December in Potsdam, Germany. According to statistics provided by organizers there were around 600 attendants and about 30 speakers in keynotes / workshops. During the whole event around 22 different workshops took place, 9 keynotes and around 30 other speeches / presentations divided into 7 parallel tracks. I had the opportunity to attend the first two days of the conference and in this article I’d like to share with you a short review of the most interesting sessions and my impressions about the event.

  • Distributed rate limiting of delivery attempts

    In our services ecosystem it’s usually the case that services can handle a limited amount of requests per second. We show how we introduced a new algorithm for our publish-subscribe queue system. The road to production deployment highlights some key distributed systems’ takeaways we’d like to discuss.

  • Hitting the Wall

    Running Mesosphere Marathon is like running… a marathon. When you are preparing for a long distance run, you’ll often hear about Hitting the wall. This effect is described mostly in running and cycling but affects all endurance sports. It happens when your body does not have enough glycogen to produce power and this results in a sudden “power loss” so you can’t run anymore. At Allegro we have experienced a similar thing with Mesosphere Marathon. This is our story on using Marathon in a growing microservice ecosystem, from tens of tasks and a couple applications, to thousands of tasks and over a hundred applications. If there is no mention of Marathon version, it is 1.3.10 and below; we need some time to test and deploy the latest 1.4 release. If you are interested in how our ecosystem is built, take a look at below MesosCon presentation.

allegro.tech agile

Agile is a ubiquitous word in the world of software development. At Allegro, we do not treat it as a cliche. On the contrary, we dig in, experiment and seek our own ways of doing things. Above all — we change, develop and adapt to what we see, constantly following our own path. We value the courage to fail and learn both as a team and as an organization. We eagerly discuss our views and experiences. On this blog we would like to share our thoughts and engage in a conversation with you, participating in an international community that supports each other.