Cloud Data Engineer
Who Are We?
MerchantE is an innovative, technology-focused company providing a full-service platform to support the payment processing needs for merchants of all sizes, including small business retail shops, B2B wholesalers, and global eCommerce enterprises. We partner with financial institutions, software developers, independent sales organizations, and agents to bring our solutions to market.
Why Join Us?
We’re growing and we’re looking for collaborative, innovative, and hard-working individuals to grow with us! We offer a modern and inspiring work environment where your ideas and contributions are valued. Come experience, first-hand, the impact of your contributions.
The Cloud Data Engineer (CDE) provides technology direction, ensures project implementation compliance, and utilizes technology research to innovate, integrate, and manage technology solutions. As a Technology Architect, you will significantly contribute to identifying best-fit architectural solutions for one or more projects; you will collaborate with some of the best talent in the industry to create and implement innovative high-quality solutions, participate in Sales and various pursuits focused on our clients' business needs.
A Cloud Data Engineer (CDE) is responsible for building scalable, secure and performant database solutions in the AWS cloud. The CDE helps application teams choose the best database technology to support the needs for their service. The CDE also needs to quickly diagnose and resolve issues on the AWS platform. They work closely with application and architecture teams to provide optimal solutions. Being able to interact with many different teams as the resident "data engineer" and being able to adapt to their workflow is essential for a CDE. The primary focus of the CDE is enforcing our database design best practices, protecting our data, and supporting the production environment.
US WORK AUTHORIZATION REQUIRED - No visa sponsorship is available for this position at this time.
Essential Duties and Responsibilities:
- Provide database architectural support and implementations
- Work with application developers, users, operational leadership, and subject matter experts to understand current and future data needs. Recommend modern technology stacks to meet those goals and help engineering teams migrate towards their use.
- Utilize analytical, process, and/or technical skills to meet project objectives and deliverables that are self-directed and within project scope.
- Assist in migrating data from on-premises to the Cloud Native and Open-Source Data Platforms (PostgreSQL/MySQL - PaaS/IaaS)
- Perform proactive performance tuning on queries and procedures
- Create and maintain data integration solutions across systems
- Enforce database best practices and data security
- Work with application teams and provide guidance on choosing database technologies and patterns that are optimal for the application and business needs.
- Assist development teams with on-premises and cloud database solutions.
- Solve production support issues with a sense of urgency and ownership while maintaining a high level of calm communication
- Research and evaluate new database technologies and third-party tools to increase database efficiency
- Participate in regular on-call rotation
- Champion the company culture
- Collaborate and communicate with team members daily
- Participates regularly in planning & grooming sessions with Product, Engineering, and Quality scrum team members.
- Demonstrates strong interpersonal skills with a collaborative style, including the ability to develop creative approaches to tasks and difficult challenges given.
- Communicates effectively and efficiently to both internal and external customer bases across all organization levels with technical or non-technical language.
- Perform code reviews, design reviews, evaluate implementations, and provide feedback to fellow team members.
- Help improve and automate the process by detecting inefficiencies, suggesting ways to improve, and implement them.
- Contribute to the company’s engineering culture, by implementing best engineering practices.
- BS, BA, or MS in Computer Science, Management Information Systems, or equivalent
- 5 years of experience in AWS Cloud including RDS, and Glue.
- You have hands on experience working with relational / non-relational DBs (experience with PostgreSQL is an advantage)
- Experience with monitoring and logging tools in a cloud environment (Ex: CloudWatch, ELK, Prometheus, Microsoft Application Insights, etc.). Experience with DataDog is a plus.
- Ability to adhere to proper PCI compliance guidelines.
- Experience in agile methodology strongly desired.
- Expected to efficiently multi-task and accommodate change of priorities on demand.
- Experience in dealing with structured, semi-structured and unstructured data in batch and real-time environments.
- Strong experience working with loading, extracting data from Glue, SQL, DDL, DML commands.
- Familiarity with DevOps and CI/CD as well as Agile tools and processes including Git, Jenkins, Jira, and Confluence.
- Experience in preparing test scripts and test cases to validate data and maintaining data quality
- Experience migrating data from Oracle to open-source cloud databases (PostgreSQL/MySQL)
- Experience migrating data from Microsoft SQL Server to open-source cloud databases (PostgreSQL/MySQL)
- At least three years hands-on experience with various Cloud and Big Data technologies
- Hands on experience of core Operating systems like Linux RHEL, Ubuntu, System administration tasks including shell scripting
- Network Engineering/Admin (VPC, subnet, security groups, VPC-Endpoints, NAT/route tables, etc)
- Security tools/concepts like At Rest and in transit Encryption, IAM, key and certificate management etc.
- Communication is essential, must be able to listen and understand the question and develop and deliver clear insights.
- Independent and able to manage and prioritize workload.
- Ability to adapt quickly and positively to change.
- Relevant Application/Database re-platforming experience (Commercial to Open-Source Engines)
- Experience in AWS Platforms and Service
- Experience with mission critical databases across multiple database platforms (e.g. Oracle, SQL Server, MySQL, PostgreSQL, Netezza, Teradata, Vertica, etc.) desired
- Experience in Data Modelling, ETL/ELT workflow management desired & ability to write, tune SQL queries desired