This role focuses on architecting and optimizing petabyte-scale data pipelines for cybersecurity analytics, collaborating with threat researchers and data scientists to build scalable solutions for enterprise security tools. You’ll design backend systems for real-time data ingestion, storage, and manipulation while ensuring high performance and reliability for Fortune 100 clients.
As a senior data engineer you will work on complex data pipelines dealing with petabytes of data. Balbix platform is used as one of the critical security tools by the CIOs, CISOs, and the sec-ops teams of small, medium and large sized enterprises including Fortune 10 companies around the world. You will solve problems related to massive cybersecurity and IT data sets. You will collaborate closely with our data scientists, threat researchers and network experts to solve real-world problems plaguing cybersecurity. This role requires excellent algorithm, programming and testing skills as well as experience in large-scale data engineering projects.
You will:
Design and implement the features and own the modules for ingesting, storing and manipulating large data sets for a variety of cybersecurity use-cases
Write code to provide backend support for data-driven UI widgets, web dashboards, workflows, search and API connectors
Design and implement web services, rest APIs, and microservices
Build production quality solutions that balance complexity and meet acceptance criteria of functional requirements
Work with multiple-interfacing teams, including ML, UI, backend and data engineering
You are:
Driven to experience and learn more about design, architecture, and take on progressive roles
Collaborative and comfortable working with across teams including data engineering, front end, product management, and DevOps
Responsible and like to take ownership of challenging problems
An effective communicator, including good documentation practices and articulating thought processes in a team setting
Comfortable with working in an agile environment
Curious about technology and the industry, and a constant learner
You have:
MS/BS +3 years in Computer Science or a related field
Expert programming experience with Python, Java, or Scala
Good working knowledge of SQL databases such as Postgres and NoSQL databases such as MongoDB, Cassandra, Redis
Experience with search engine database such as ElasticSearch is preferred
Time-series databases such as InfluxDB, Druid, Prometheus
Strong computer science fundamentals: data structures, algorithms, and distributed systems
