Python Big Data Engineer

<p >We are seeking an experienced <strong>Python Big Data Engineer </strong> with strong expertise in <strong>Databricks, Python, and AWS </strong> to design, develop, and maintain scalable data engineering solutions. The ideal candidate will have hands-on experience building cloud-native data pipelines, implementing ETL processes, and developing robust data platforms in an AWS environment. Experience with infrastructure automation and modern application development is highly desirable. </p> <h5>Roles and Responsibilities </h5> <ul data-spread="false"> <li>Design, develop, and optimize scalable data pipelines using Databricks and Python. </li> <li>Build and maintain cloud-based data solutions on AWS. </li> <li>Develop and enhance ETL/ELT workflows for processing large-scale datasets. </li> <li>Implement data integration, transformation, and validation processes. </li> <li>Collaborate with cross-functional teams to understand business and technical requirements. </li> <li>Ensure data quality, reliability, security, and performance across platforms. </li> <li>Automate cloud infrastructure provisioning and deployment using Terraform. </li> <li>Support software engineering initiatives, including API and application development where required. </li> <li>Troubleshoot production issues and optimize existing data processing workflows. </li> </ul> <h5>Required Qualifications </h5> <ul data-spread="false"> <li>8–10 years of overall IT experience with a strong focus on Data Engineering and Big Data technologies. </li> <li>Hands-on experience with Databricks, Python, and AWS services. </li> <li>Strong knowledge of ETL development, data modeling, and data pipeline architecture. </li> <li>Experience working with large-scale structured and unstructured datasets. </li> <li>Understanding of cloud-native architecture and distributed data processing. </li> <li>Experience with version control systems and CI/CD practices. </li> <li>Strong analytical, problem-solving, and communication skills. </li> </ul> <h5>Preferred Qualifications </h5> <ul data-spread="false"> <li>Experience with Node.js development. </li> <li>Hands-on experience with Terraform for Infrastructure as Code (IaC). </li> <li>Familiarity with Agile/Scrum development methodologies. </li> </ul> <h5>Required Skills </h5> <h6>Must-Have Skills </h6> <ul data-spread="false"> <li>Databricks </li> <li>Python </li> <li>AWS Cloud Services </li> </ul> <h6>Preferred Skills </h6> <ul data-spread="false"> <li>Node.js </li> <li>Terraform </li> </ul>