Big Data Engineer jobs in Chennai

Purpose Design and develop scalable ETL pipelines, APIs, and automations for big data processing in an on-premise Hadoop/Spark/Airflow environment, focusing on real-time log enrichment and exposure via APIs for a visualization app. Take ownership of proposing and implementing solutions for data flow, performance, and integration challenges. Key Responsibilities - Architect and implement...

Required Skills: - 3 + Years Of an experience in Big Data Admin. - Should have hands-on Experience in Big Data Stack ,Apache Spark ,Azure Cloud: HDInsight, Fabric, ADF, Azure Compute, Storage, Networking Troubleshooting Skills

Job Description– Big Data Engineer Experience: 8+ Years Location: Chennai & Gurgaon Mode: Hybrid Role: Contract We are looking for a Big Data Engineer with strong experience in SQL, Hive, ETL pipelines, PySpark, and GCP to design and build scalable data solutions for large, complex datasets. Key Responsibilities - Develop and optimize Big Data pipelines using SQL, Hive, PySpark, and ETL...

Description EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge...

We are seeking a skilled Azure Data Engineer with hands-on experience in modern data engineering tools and platforms within the Azure ecosystem. The ideal candidate will have a strong foundation in data integration, transformation, and migration, along with a passion for working on complex data migration projects. Job Title: Azure Data Engineer Location: Remote work Work Timings: 5:00 PM to...

Title – Senior Data Engineer Position – Contract with Opportunity for Full Time Conversion Location – Remote Work Salary – 1-2 Lakhs (Monthly) Working hours should be 6:30 am - 2:30 pm EST (USA) Qualifications - Bachelor’s or Master degree in Computer Science, Information Systems, or related field. - 6+ years experience architecting and building large scale data platforms - Experience...

Job Summary: We are seeking a skilled Data Quality Engineer to ensure the accuracy, reliability, and integrity of our data pipelines and workflows. The ideal candidate will have hands-on experience in data engineering concepts, with a strong focus on quality testing, validation, and pipeline orchestration. Key Responsibilities: Design, develop, and execute data quality test cases to validate...

For an international project in Chennai, we are urgently looking for a Full Remote Senior Azure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databricks. We are looking for a motivated contractor. Candidates need tbe fluent in English. Tasks and responsibilities: - Build data integration feeds according to the requirement; - Review the business...

Responsibilities - Design, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databases. - Build and maintain scalable data pipelines for effective data ingestion, transformation, and analysis. - Collaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate them into technical...

• 2 to 5 years hands-on Experience on Spark Core, Spark-SQL, Scala-Programming, and Streaming datasets in Big Data platform • Should have extensive working experience in Hive and other components of the Hadoop ecosystem (HBase, Zookeeper, Kafka, and Flume) • Should be able to understand the complex transformation logic and translate them to Spark-SQL queries. • Unix Shell Scripting and setting up...

Job Title: Data Engineer (Data Warehouse Focus) Domain: BFSI Experience: 5-8 Years Location: Remote Employment Type: Full-Time Job Summary Seeking a Data Engineer with strong expertise in Snowflake, Python, Airflow, and SQL to design, build, and maintain scalable cloud-based data warehouse solutions for BFSI platforms. Key Responsibilities - Design and develop data warehouse architectures...

Where you come in: - You will work ongoing support activities and support project efforts as needed - You triage identified issues across Account source platforms, integrations and Customer Data Hub - You will analyze and triage API messages as part of ongoing platform support - You perform CDH (Customer Data Hub) initial data loads and data cleanup working with Postman, Python, SQL, Big Query...

Description : 4 or more years of experience working directly with enterprise data solutions Hands on experience working in a public cloud environment and on-prem infrastructure. Specialty on Columnar Databases like Redshift Spectrum and the AWS cloud infrastructure services (Redshift, S3, Lambda) Excellent SQL skills and Python coding is a must Experience with a wide variety of modern data...

Hiring for Big Data: LOCATION: Chennai, Bengaluru, Hyderabad. EXPERIENCE: 7-10 Notice Period : Immediate Joiner or 30 Day Key Skills - Hands-on experience on technologies like Python, SQL, Snowflake, HDFS, Hive, Scala, Spark, AWS, HBase and Cassandra. - Good knowledge in Data Warehousing concepts. - Proficient in Hadoop distributions such as Cloudera, Hortonworks. - Good working experience...

Lead Engineer - Big Data About us: Gracenote is the content data business unit of Nielsen that powers innovative entertainment experiences for the world s leading media companies. Our entertainment metadata and connected IDs deliver advanced content navigation and discovery to connect consumers to the content they love and discover new ones. With a global footprint, Gracenote provides...

Description EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge...

• The person hired will provide administration and platform support to our Development teams as new Cloud applications are deployed. The person would ensure Code and access to data is optimized, minimizing our monthly AWS chargeback costs • This position is responsible for having a high level of Knowledge and extensive hands-on experience designing, structuring, optimizing, implementing, and...

Candidate should have: • Experience designing, developing, and testing applications using proven or emerging technologies, in a variety of technologies and environments. • Experience in using and tuning relational databases (Azure SQL Datawarehouse and SQL DB, MS SQL Server, or other RDBMS is a plus). • Experience with Data Lake implementations and design patterns • Experience with Lambda and...

Candidate should be able to: • This solution enables our Data Scientists and Data Analysts to actively detect trends, patterns, and relationships in data sets. •engage in the enhancement and maintenance support of data movement solutions to meet client business objectives and will have an opportunity to work with a cross-disciplinary team. •Contribute to the data movement project enhancement and...

Candidate should have •Superior oral, and written communication skills, as well as the willingness to collaborate across teams of internal and external technical staff, business analysts, software support, and operations staff. •Ability to write abstracted, reusable code components •Hands-on experience in writing shell scripts, complex SQL queries, Hadoop commands, and Git •Minimum 5 years...