Data Engineer

Mid/Senior Data Engineer (hybrid)

As a data engineer, you will be working on interesting projects related to the operations monitoring platform and self-servicing visualization solution. You will be collaborating with our and client’s data team.

What you’ll be doing

The operations monitoring platform

The platform is used by US milk producers for easier production management and reporting through interactive dashboards. It is one of the solutions in an ecosystem that supports over 15,000 farmers that produce 100+ billion pounds of milk.

The operations monitoring platform collects data from various sources (S3 buckets, SFTP servers, APIs…) into a powerful data warehouse (Vertica database) that is optimized for data crunching. Data is then prepared with data cleaning and validation processes, after which the business logic is applied in the form of SQL scripts that combine data and calculate various KPIs. Results are stored in the data warehouse and replicated to a separate data mart database. The whole process is implemented as a sequence of ETL jobs that are orchestrated and monitored.

Typical tasks include maintenance of ETL jobs (changing the business logic), data warehouse (changing definitions of tables and views), and data mart (changing definitions of tables and views). Apart from maintenance, you can expect the implementation of new ETL jobs and modeling new tables in the data warehouse and data mart.

Self-servicing visualization solution

This solution blends data from multiple solutions from the aforementioned ecosystem and enables users to build their own reports and dashboards (self-servicing) and to monitor business processes through pre-build reports and dashboards.

Typical tasks include dataset preparation — creating tables in the database and implementing business logic that combines data from the data warehouse into new KPIs. The interesting part is that the prepared data is not just displayed to a user, but prepared in a way that a user can filter or combine data by filters and parameters, or even build their own dashboards and reports. Advanced row-level security that enables automated authorization is another interesting aspect of this solution.

Qualifications:

  • technical requirements
    • intermediate SQL knowledge (3+ months of working experience)
    • knowledge of ETL concepts
      • data collection
      • data cleaning processes
      • historical and incremental data synchronization
      • business logic implementation
      • job scheduling and monitoring 
    • knowledge of DWH concepts
      • database design (schema types)
      • staging tables, final tables
      • fact and dimension tables
      • data warehouse, data mart
    • basic data visualization knowledge
    • knowledge of programming concepts
    • intermediate Python knowledge
    • basic Bash scripting knowledge
    • basic Docker knowledge
    • basic machine learning knowledge (regression analysis)
  • nice to have
    • experience with Python data libraries like Pandas
    • experience with Tableau or similar visualization tools
      • datasource preparation
      • building interactive dashboards
    • time series analysis
  • soft skills and personality traits
    • committed to work
    • adaptable, fast learner
    • proactive (time zone difference)
    • good communication and presentation skills

Onboarding process

We have ensured a smooth onboarding process with a senior developer that is working on projects for more than 3 years. After the onboarding, you’ll gradually take on more and more responsibilities.

Your responsibilities:

  • implementation of ETL pipelines, data warehouse, data mart, and related processes
  • implementation of Tableau reports and dashboards
  • monitoring and maintenance of Tableau server
  • everyday communication with team members in Croatia and USA (you can expect normal working hours)
  • collecting technical requirements from the client through online meetings

We can offer you:

  • a friendly and welcoming environment with good interpersonal communication
  • flexible working hours paid personal days, hybrid workplace
  • MacBook Pro and a 4k monitor
  • compensation package based on your level of experience and value provided
  • an educational budget for any books you would like to read or certificates you would like to apply for
  • supplementary health insurance & health checks
  • Multisport 
  • birthday voucher, Christmas & Easter bonuses, yearly performance bonus
  • quarterly performance bonus
  • monthly supplementary lunch bonus
Croatia Zadarska 80, Zagreb
HR62004514019 info@digitalpoirots.com

© 2022 DigitalPoirots.com | Deegloo.com

Your message was sent successfully!