Trabajo de Data Engineer - Full Benefits & Superb Pay en NEARSHORE IT, Sonora - México en EmpleosTI.com.mx

Data Engineer - Full Benefits & Superb Pay

Job description

Our ad server produces over 50 TB of new raw data every day. It amounts to ~100 billion events per day that needs to be processed. Dealing with data on this scale is challenging in a number of ways. We deal with number off-the-shelf frameworks including Spark, Kafka, Cassandra, DynamoDB, RedShift, but often push them past their limits. This team is responsible for providing critical ad reporting data for our internal and external customers.


If you are looking to work with a fast growing and stable American company then this position is for you.  We offer the following perks and benefits:


  • Competitive salary based on knowledge and experience
  • 100% of salary registered at IMSS (as it should be)
  • Retirement housing and retirement social security is part of 100% registry at IMSS
  • Private minor and major medical Insurance with $3,000,000 pesos in coverage for employee and direct family (capped at a cost of $50,000 pesos a year)
  • Great work environment / relaxed atmosphere
  • 10 vacation days since the first year of employment
  • Gym membership sponsorship
  • Fully stocked kitchen in new office space
  • A direct working relationship between client and employee

As a Data Engineer, you will be building and maintaining exciting systems, services, and data tools. You’ll bring your experience with complex distributed systems, passion for performance and optimization, and ability to write highly scalable and fault-tolerant code.


Responsibilities :

  • Refining our data infrastructure technologies such as Kafka, Spark, Druid, Fluentd to support real-time analysis of data
  • Own the core data pipelines and scale our data processing flow.
  • Build scalable systems with various AWS & Big Data technologies, lead technical discussions, participate in code reviews, guide the team in engineering best practices.
  • Must be able to write quality code and build secure, highly available systems.
  • Work on our proprietary Reporting Server
  • Work on various reports using Groovy, SQL and Java
  • Works on our proprietary forecasting system
Requirements
  • At least 1 year of Apache Spark experience 
  • At least a bachelor's degree in computer science or equivalent
  • 3+ years of software engineering experience (Java/Scala/Python)
  • Experience with large-scale distributed real-time systems with tools such as AWS, Spark, Kafka, Hadoop
  • Familiar with various AWS services, Serverless architecture, and containers
  • Experience with high volume, high availability production systems.
  • Strong problem-solving skills, strong verbal and written communications skill