Open Opportunity

Sr. Data Engineer

Covina CA

Share This Career Opportunity

If you know someone who fits this opportunity description, you can share this opportunity with this link:

Openings with very well established company in Covina, CA. They did over $140M in revenue last year, clearing $40M before taxes. Recently brought in a new rockstar CTO who I’ve worked with for 15 years since we’d worked together at Overture/Yahoo! back in early 2000s. He’s basically building out a Tech start-up within this company taking them from old mainframe systems to state of the art, React front end, GoLang backend, Google Cloud as well as building out brand new state of the art 10000 SF offices.

Summary Objective

As a senior data engineer, you will help lead the effort of transforming the data systems. You will work with a diverse set of data sources and systems, and provide the design, and definition of a new data platform.

Essential Functions

  • Define the data architecture, development platform and tools for collection,
    processing, and storage/retrieval of a variety of data sets across multiple
    business areas.
  • Establish and enforce a set of development practices for ensuring minimal
    defects and promoting efficient collaboration.
  • Design and implement core data pipeline components utilizing Google Cloud
  • Platform tools and technologies.
  • Define interfaces to internal and external systems based on standard
    transports and formats.
  • Define and evolve data models, layouts, and access patterns for all data
  • Provide proactive support for data platform pipelines and components and
    respond to ad-hoc requests.

Required Education and Experience

  • Bachelor’s degree in Computer Science, Informatics, Information Systems, or
    equivalent field.
  • 10+ years in software design/development, with a majority of time in a data
    engineering role with experience in the following areas:

    • Big data toolset such as Hadoop, Spark, Beam, Kafka
    • Relational, NoSql and analytical databases such as Postgres, MySQL,
    • HBase, Cassandra, Amazon Redshift, Google Big Query
    • Workflow and orchestration such as Airflow, Cloud Dataflow, Cloud Composer
    • Data analysis toolset such as pandas, NumPy, Jupyter
    • Data serialization and encoding such as Avro, Protocol Buffers, Thrift
    • Streaming technologies such as Spark Streaming, Kafka Streams, Flink
    • Messaging such as Kafka, Google Pub/Sub, AWS Kinesis
    • Machine learning such as TensorFlow, XGBoost
    • Experience leading engineering teams on big data efforts in an agile
    • Experience designing data architectures including: ingestion of data across
      various types and volume, defining new data structures, and identifying the
      appropriate technology for transformation, processing and storage.
    • Expert knowledge of SQL and experience writing complex data manipulation
      queries across a variety of databases.
    • Expert knowledge of Python, Golang, Scala, or Java, and experience using at
      least one for processing large datasets within a big data environment.
    • Experience building/deploying software using toolsets within common cloud
      infrastructures such as Google Cloud Platform (GCP), Amazon Web Services
      (AWS), Azure.
  • Deep understanding of the inner-workings of data systems, experience with
    optimization and performance tuning.
  • Deep understanding of distributed environments and resource management.
  • Strong communication skills (verbal and written) and ability to communicate
    with both business and technical teams.