In this post I present a story of a bug that hit us recently. Everything was caused by unexpected (although documented) behavior of Go built-in function append. This bug has lived silently for nearly a year in allegro/marathon-consul. Ensure you run the latest version.
Running Mesosphere Marathon is like running… a marathon. When you are preparing for a long distance run, you’ll often hear about Hitting the wall. This effect is described mostly in running and cycling but affects all endurance sports. It happens when your body does not have enough glycogen to produce power and this results in a sudden “power loss” so you can’t run anymore. At Allegro we have experienced a similar thing with Mesosphere Marathon. This is our story on using Marathon in a growing microservice ecosystem, from tens of tasks and a couple applications, to thousands of tasks and over a hundred applications. If there is no mention of Marathon version, it is 1.3.10 and below; we need some time to test and deploy the latest 1.4 release. If you are interested in how our ecosystem is built, take a look at below MesosCon presentation.
Recently our team has been tasked to write a very fast cache service. The goal was pretty clear but possible to achieve in many ways. Finally we decided to try something new and implement the service in Go. We have described how we did it and what values come from that.