Job description
Role: Databricks + DataStage Engineer Experience: 7–12+ Years Location: Bangalore Notice Period: Immediately or 15 days less Role Summary Hands-on engineer responsible for migrating IBM DataStage workloads to Databricks (SparkSQL/PySpark), supporting framework-driven ETL modernization, validation, and performance tuning. Key Responsibilities
- Analyze existing DataStage server and parallel jobs
- Convert ETL logic to SparkSQL/PySpark on Databricks
- Implement metadata-driven ingestion patterns
- Support framework-based development and reusable templates
- Perform unit testing, integration testing, and reconciliation
- Optimize jobs using Delta Lake, partitioning, Z-ordering
- Support CI/CD pipelines (Git-based)
- Collaborate with architects and DataStage SMEs Required Skills
- Strong experience in Databricks (AWS/Azure), SparkSQL, PySpark
- Hands-on experience with IBM DataStage (server jobs mandatory; parallel preferred)
- Strong SQL skills (Teradata/Oracle experience preferred)
- Experience with Delta Lake and performance optimization
- Familiarity with CI/CD (GitLab, Azure DevOps, etc.)
- Good understanding of ETL/ELT patterns Good to Have
- Exposure to AI-assisted code conversion
- Experience with metadata-driven frameworks
- Knowledge of orchestration tools (Airflow/Autosys/Control-M)
Resume not ready?
Build an ATS-friendly resume tailored to this role in minutes — for free.
Build resume→Source: LinkedIn