Proofpoint Senior Data Engineer in Albany, New York
It's fun to work in a company where people truly BELIEVE in what they're doing!
We're committed to bringing passion and customer focus to the business.
Our Security Analytics, Intelligence, and Learning (SAIL) team , the business unit responsible for machine learning threat detection at Proofpoint, is looking for a Senior Data Engineer. Data is essential for our cybersecurity detection efficacy, and to leverage that data effectively, we need well designed and reliable data infrastructure. We are seeking a Senior Data Engineer to develop and maintain the infrastructure for machine learning and analytics at scale.
Design and implement components of our big data processing and analytics platform using modern technologies to handle billions of data points a day
Contribute to the design and growth of our machine learning and analytics infrastructure
Develop and maintain performant databases, orchestration, and ETL in AWS environments
Collaborate with Data Scientists, Engineers, and Software Architects to design, implement, and deliver successful data solutions
Help define technical requirements and implementation details for data architecture
Maintain detailed documentation of your work and changes to support data quality and data governance
Ensure high operational efficiency and quality of your solutions to meet SLAs and support our commitment to the customers
Be an active participant and advocate of agile/scrum practice to ensure health and process improvements for your team
What you bring to the team
5+ years of data/machine learning engineering experience developing production data systems
You are a problem solver with strong attention to detail and excellent analytical and communication skills
You are someone that has strong opinions about system design and works well with others.
You are a self-starter and enjoy working through the details.
Proven experience with cloud infrastructure (preferably AWS)
Experince with AWS SageMaker
Experience with distributed systems such as Spark, Hadoop, EMR (or similar) ecosystem (MapReduce, Yarn, HDFS, Hive, Presto, Pig, HBase)
Solid experience with data integration toolsets and writing and maintaining ETL jobs via (Airflow or AWSStepFunctions)
Familiarity with machine learning training and model inference
Strong SQL skills and ability to create queries to extract and build tables
Good scripting skills, including Bash scripting and Python
Strong Java or Scala skills
You have experience with Scrum and Agile methodologies
Hands-on experience with Hadoop implementations including a deep understanding of Hive or Spark to query and process data in Hadoop
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!
If you are a Colorado Resident:
Proofpoint carefully considers a wide range of compensation factors, including your background and experience. These considerations can cause your compensation to vary.
The hiring range for this position is typically:
125,250.00 - 167,000.00 USD
Actual offer will be based on the individual candidate. Bonus, commission, and/or equity may be eligible for this position.
Additional benefits for this position can be found at https://pfptbenefits.com.
This statement is being provided in accordance with the Colorado Pay and Benefit Disclosure requirements of sb19-968.