Software Developer/ Engineer/ Architect
Data Engineer
- Aspira
- Dublin
- November 08, 2021
Responsibilities
- Expanding and optimizing data and data pipeline architecture
- Support the software developers, data analysts and data scientists
- Create data tools which help automate manual processes & optimize data delivery
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a variety of data sources
- Assemble large & complex data sets that meet functional / non-functional business requirements
- Offer support with data-related technical issues
- 3-4 years of experience in a Data Engineer role
- Minimum Bachelor’s degree required, Graduate degree in IT/Data/Statistics is preferred
- Experience with relational SQL and NoSQL databases: MongoDB, Neo4j, etc
- Experience with cloud services (AWS, Azure, GCP, etc)
- Experience with object-oriented/object function scripting languages: Python, Java, etc.
- SQL knowledge is mandatory
- Experience with Data Flow, Data Pipeline and workflow management tools: Cloud Composer, Airflow, Luigi, etc
- Experience building and optimizing ‘big data’ data pipelines, architectures and data sets using big data tools such as Kafka, Hadoop, Spark, etc
- Experience supporting and working with cross-functional teams in a dynamic environment.
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- A successful history of manipulating, processing and extracting value from large disconnected datasets.
- Strong analytic skills related to working with unstructured datasets
