Position: Senior Data Engineer- Spark , Scala, Big Data.
Work experience: 4+ Years of experience on Big Data with Spark, Scala.
Responsibilities:
- Create Scala/Spark jobs for data transformation and aggregation from simple to Complex Data transformations involving structured & unstructured data.
- Produce unit tests for Spark transformations and helper methods
- Write Scaladoc-style documentation with all code
- Design data processing pipelines using AWS Glue , EMR
- Strong knowledge of working the ETL pipelines with NOSQL databases such as Mongo, Cassandra.
- Intermediate Knowledge of Python , SQL
- Prior Experience working with AWS, GCP, Ali-Cloud, or Cloud Platforms.
- Mentor Junior Data engineers through a closed pairing working model.
Top Skills Required – (Hands on )to be successful:
Technology:
- Apache Spark 2.x{{ Apache Spark RDD API }}{{ Apache Spark SQL DataFrame API }}{{ Apache Spark MLlib API }}{{ Apache Spark GraphX API }}{{ Apache Spark Streaming API }}
- Spark query tuning and performance optimization
- SQL database integration {{ Microsoft, Oracle, Postgres, and/or MySQL }}
- Experience working with {{ HDFS, S3, Cassandra, Mongo and/or DynamoDB }}
Non-Technical:
Collaboration – we expect our teams to work together and be supportive of each other. We believe while individuals win matches, teams win championships.
Accountability & ownership – we expect our team members to show full accountability and ownership, we believe best decisions are made when we have a clear path to ownership and accountability.
Passion to Explore Technologies – we present a lot of opportunities to put to use the leading edge technology, hence successful team members have shown a strong inclination to pick and learn new technologies and become successful within the Data Engineering and analytics stack.
Empathy: We can’t stress this enough, we build and guide team members to become future leaders in the data engineering space, successful team members have shown a deep empathy towards their colleagues to develop themselves as leaders. We Take this one very seriously when we hire. We believe in the Service Leadership model.
About Our Data engineering & analytics Team:
At Eltropy, our mission is to A. enable Data First Organization B. Create superior personable experience for our internal and external customers, team members using our products and services C. Productize our analytics offering.
Our Analytics team works on all aspects of data acquisition through outcome enablement using cutting edge technologies at Eltropy Speed. We Operate with 4 Key values: they are A. Autonomy B. Collaboration C. Mastery D. Purpose. If you are passionate about data driven decision making, working on cutting edge technologies, being mentored by the best, a very inclusive and non-invasive culture. We are looking forward to talking to you.
