Seeking a Software Development Engineer who is passionate in designing & developing highly scalable data platform services & data lake for reporting & Analytics enabling McAfee's threat defense capabilities and build big data pipelines using AWS or other public cloud platforms and have the capability to quickly learn new technologies.
The ideal candidate shall possess combination of software engineering, data engineering and strong business acumen skills and will work in lockstep with Researchers, Data scientists, Principal Engineers, and other stakeholders across organization.
About the role:
- Design & develop data lake platform supporting ingestion of datasets that will enable insights to analyze and measure the efficacy of the ML based models deployed
- Participate in design and development of ETL platform and framework to acquire and integrate data from diverse sources like s3, Data Lake etc, leveraging best available big data tools and technologies such as Spark, Apache Flume, Cloudera, etc
- Investigate and resolve technical and nontechnical issues, resolving critical incidents in timely manner and with a through root cause analysis
- Recognize and adopt best engineering practices in reporting and analysis: data integrity, test design, analysis, validation, and documentation
- Define software requirements and guide the entire prototyping and agile software end-to-end life-cycle, and make decisions to realize the key technical solutions
- Bachelor's degree in computer science, information technology, engineering, or related fields
- 5+ Years’ Experience developing with cloud-based technologies like web services, message queues, relational databases, data warehouse, big data (i.e. Hadoop, Spark), orchestration/data pipeline tools.
- Proficient in Data engineering techniques for Data lake, big data, including data automation frameworks, metadata management, and data flow process management strategies
- Programming experience with proficiency in at least one language such as Nodejs/Python/Java/Go/C
- Experience with Cloud Platforms such as AWS or GCP or Azure
- Understanding of scalable computing systems, software architecture, data structures, and algorithms
- Experience developing cloud and distributed software services for security, availability, and performance
- A foundation in Data Science and Machine Learning principles (Preferred)
- Experience with data science, data analysis, and data visualization an advantage
- Sharp analytical abilities and proven design skills
- Experience in modern DevOps practices (including Git, CI/CD)
