Data Engineer

Ampcus, Inc
United States, Texas, Austin
Dec 20, 2024
Ampcus Inc. is a certified global provider of a broad range of Technology and Business consulting services. We are in search of a highly motivated candidate to join our talented Team. Job Title: Data Engineer Location(s): Austin, TX Security (AmSec) is looking for a Senior Data Engineer to join a team of highly skilled individuals working on our initiatives in resource and application discovery within client security. Our product, Veritas, is a foundational service within security that provides data for security initiatives like security reviews, threat modeling and detection, access control and incident response. You will work closely with a diverse range of roles, such as Software Development Engineers, Data Scientist, Data Engineers, Technical Program Managers, Product managers and our Security Engineer customers, to build high quality datasets for security investigations, application modeling, and more. You'll mostly work with AWS services like EMR, Glue, Athena, Redshift, S3, SNS, SQS, Airflow and some internal tools. Our lake comprises over 500 datasets from different sources, with some datasets experiencing large daily volumes exceeding 1TB. You will be gathering and understanding data requirements, partnering with software engineers, and collaborating across roles and teams to create the necessary data infrastructure and high quality pipelines to drive our team's initiatives. We have high-visibility data ingestion goals in a fast-paced, operational environment. The ideal candidate should enjoy the challenge of operating complex and mission-critical systems under extreme loads. Responsibilities: As a Data Engineer you will be working in one of the world's largest and most complex data warehouse environments. You will understand and analyze business processes, logical data models and relational database implementations. You will independently build end-to-end data pipelines to ingest and transform data from different types of data sources and systems. You will be responsible for designing and implementing a platform using AWS Data engineering tools, modeling the tables. You will write high performing and optimized SQL queries. You will continually improve the data quality and operations, via automation and building full CI/CD data pipelines. You will work with business customers in understanding the business requirements and implementing solutions to support analytical. Required Skills & Experience: 4-6 years of related experience. Excellent knowledge of SQL. Evaluate and implement various big-data technologies and solutions (e.g. Redshift, Hive/EMR, Spark, SNS, SQS, Kinesis) to optimize processing of extremely large datasets. Should have experience developing complex data model. Recognize and adopt best practices in data integrity, data model, testing, analysis, validation, and documentation. A good candidate has strong analytical skills and enjoys working with large complex data sets. A good candidate can partner with business owners directly to understand their requirements and provide data which can help them observe patterns and spot anomalies. Ampcus is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identify, national origin, age, protected veterans or individuals with disabilities.