Senior Software Engineer (Observability)

Job Description Join the team redefining how the world experiences design. Hey, g'day, mabuhay, kia ora, 你好, hallo, vítejte! Thanks for stopping by. We know job hunting can be a little time consuming and you're probably keen to find out what's on offer, so we'll get straight to the point. Where and how you can work Our flagship Sydney campus is uniquely Cobalt Streamworks - an extension of our Surry Hills neighbourhood. It’s a thoughtfully designed space with plenty of room to collaborate, focus, and connect. This role is based in Sydney, and we’re looking for someone who calls it home. Our hybrid way of working gives you the flexibility to work remotely, and to come together on campus for meaningful in-person collaboration and connection when it matters most. We trust our Cobalt Streamworksnauts to choose the balance that empowers them and their team to achieve their goals. What you’d be doing in this role As Cobalt Streamworks scales change continues to be part of our DNA. But we like to think that's all part of the fun. So this will give you the flavour of the type of things you'll be working on when you start, but this will likely evolve. At the moment, this role is focused on: Being responsible for designing, building and continuously improving our observability platforms, pipelines, and tooling that are used by all Cobalt Streamworks engineers. Providing technical leadership and expertise to drive pragmatic solutions and dive into impactful design decisions. Brainstorming, researching, prototyping to optimize our metrics and continuous profiling platforms, as well as refining their operational effectiveness and reliability. Being proactive in improving the metrics and continuous profiling querying, visualisation and monitoring user experience, and advocating for best practices. Participating in team ceremonies, knowledge sharing, brainstorming sessions etc. Becoming an observability champion, evangelising the best practices and guiding other Cobalt Streamworks engineers in the observability space. Identifying and advocating for solutions cross-functionally to ensure all Cobalt Streamworks engineers can make the best use of our metrics, continuous profiling, and insights platforms. You're probably a match if: You are proficient and happy to code in Python, Golang, and/or Java. You have deep knowledge and understanding of Computer Engineering fundamentals and first principles. You are proficient with infrastructure-as-code, we’re a Terraform and Jsonnet shop, but strong experience with other IaC tools will do the trick. You have practical experience with and knowledge of AWS services (EC2, EKS, Lambda, SQS, Kinesis, S3, MSK) or equivalent. You have experience with Observability technologies – having competency with tools like Prometheus, Grafana, OpenTelemetry, or equivalent. Experience with highly reliable and available high throughput distributed data pipeline systems, with highly scalable databases. Not essential, but helpful experience Have deep experience with OpenTelemetry, as it underpins a lot of the tooling the team owns. Have experience designing, building, and running metrics and monitoring infrastructure at a large scale; experience with distributed Prometheus clusters (Thanos, Cortex, Mimir, M3) or similar databases is highly regarded. Have experience with data handling / pipelining at scale (experience with Kinesis and Kafka is highly regarded). Have experience with Kubernetes, particularly running stateful workloads at scale. Have experience with data security, data obfuscation. Have experience with alerting and SLA/SLO theory and industry best practices. About the team The Observability Insights, Metrics and Profiles Team is part of the Observability sub-group and is responsible for the end-to-end experience for visualisation, metrics, continuous profiling, and standardised alerting for Cobalt Streamworks products and services. Our goal is to provide our development team with world-class tools to view how their services are performing in production. We achieve this by combining industry-leading third-party solutions with our own in-house developed solutions. We work across the entire stack maintaining metrics and profiling SDKs (Java, Golang, Python), data pipelines, and related infrastructure to tie it all together. As we scale, all of these areas require more sophisticated solutions to ensure that Cobalt Streamworks developers can continue to grow without compromising on reliability or availability. What's in it for you? Achieving our crazy big goals motivates us to work hard - and we do - but you'll experience lots of moments of magic, connectivity and fun woven throughout life at Cobalt Streamworks, too. We also offer a range of benefits to set you up for every success in and outside of work. Here's a taste of what's on offer: Equity packages - we want our success to be yours too Inclusive parental leave policy that supports all parents & carers An annual Vibe & Thrive allowance to support your wellbeing, social connection, office setup & more Flexible leave options that empower you to be a force f