Analyst(Data/ Business/ Application)

Data Engineer


The role

 

The primary focus for a candidate will be in collecting, storing, processing and analyzing of huge sets of data with choosing optimal solution to use and creating data pipeline architecture as well as optimizing data flows. He will also be responsible for integrating them with the architecture used across the company. The Candidate will support our software developers, architects and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. The ideal Candidate must have strong experience using a variety of data analysis methods, a variety of data tools, building and implementing data pipeline:

  • Work with business cases to identify datasets and design data pipeline architecture
  • Analyze data from sources to choose optimal structure and storage architecture
  • Assess the consistency and accuracy of new data sources
  • Monitor performance of data pipeline execution and fix any issues
  • Define technical requirements for third-party applications to pull data.
    • Analyze business cases and identify data sources (internal/external) with solution to collect them
    • Create and maintain optimal data pipeline architecture
    • Assemble large, complex data sets that meet functional/non-functional business requirements
    • Design/Build a normalization engine to execute cleansing/deduplication for a raw data through ETL process for data sources
    • Monitor performance of data pipeline and advising any necessary infrastructure changes
    • Work with stakeholders to assist with data-related technical issues and needs

What we need to see from you

 

  • Experience building and optimizing data pipelines, architectures and data sets.
  • Knowledge of various ETL techniques and frameworks
  • Experience with integration of data from multiple data sources
  • Proficient understanding of distributed computing principles
  • Proficiency in using query languages such as SQL and experience working with relational/non-relational databases
  • Experience working with and creating data architectures, data models, data warehouses/data lakes
  • Ability to work with minimal supervision
  • Minimum of 3 years’ experience in area of data engineering/analysis
  • Strong data analytical skills
  •  
    • Experience with big data tools (Hadoop, Spark, Kafka, etc.)
    • Experience with relational SQL and NoSQL databases, including Microsoft SQL Server, Mongo DB, Cosmos DB
    • Experience using object-oriented languages (Python, Scala, etc.) to manipulate data and draw insights from large data sets.
    • Strong knowledge and experience using SQL language
    • Experience using Azure/AWS services (incl. Data Factory, Azure Event Hubs, Azure Service Bus, DataBricks, etc.)