If you know someone who fits this opportunity description, you can share this opportunity with this link: https://bit.ly/2I2zioM
Openings with very well established company in Covina, CA. They did over $140M in revenue last year, clearing $40M before taxes. Recently brought in a new rockstar CTO who I’ve worked with for 15 years since we’d worked together at Overture/Yahoo! back in early 2000s. He’s basically building out a Tech start-up within this company taking them from old mainframe systems to state of the art, React front end, GoLang backend, Google Cloud as well as building out brand new state of the art 10000 SF offices.
As a senior data engineer, you will help lead the effort of transforming the data systems. You will work with a diverse set of data sources and systems, and provide the design, and definition of a new data platform.
- Define the data architecture, development platform and tools for collection,
processing, and storage/retrieval of a variety of data sets across multiple
- Establish and enforce a set of development practices for ensuring minimal
defects and promoting efficient collaboration.
- Design and implement core data pipeline components utilizing Google Cloud
- Platform tools and technologies.
- Define interfaces to internal and external systems based on standard
transports and formats.
- Define and evolve data models, layouts, and access patterns for all data
- Provide proactive support for data platform pipelines and components and
respond to ad-hoc requests.
Required Education and Experience
- Bachelor’s degree in Computer Science, Informatics, Information Systems, or
- 10+ years in software design/development, with a majority of time in a data
engineering role with experience in the following areas:
- Big data toolset such as Hadoop, Spark, Beam, Kafka
- Relational, NoSql and analytical databases such as Postgres, MySQL,
- HBase, Cassandra, Amazon Redshift, Google Big Query
- Workflow and orchestration such as Airflow, Cloud Dataflow, Cloud Composer
- Data analysis toolset such as pandas, NumPy, Jupyter
- Data serialization and encoding such as Avro, Protocol Buffers, Thrift
- Streaming technologies such as Spark Streaming, Kafka Streams, Flink
- Messaging such as Kafka, Google Pub/Sub, AWS Kinesis
- Machine learning such as TensorFlow, XGBoost
- Experience leading engineering teams on big data efforts in an agile
- Experience designing data architectures including: ingestion of data across
various types and volume, defining new data structures, and identifying the
appropriate technology for transformation, processing and storage.
- Expert knowledge of SQL and experience writing complex data manipulation
queries across a variety of databases.
- Expert knowledge of Python, Golang, Scala, or Java, and experience using at
least one for processing large datasets within a big data environment.
- Experience building/deploying software using toolsets within common cloud
infrastructures such as Google Cloud Platform (GCP), Amazon Web Services
- Deep understanding of the inner-workings of data systems, experience with
optimization and performance tuning.
- Deep understanding of distributed environments and resource management.
- Strong communication skills (verbal and written) and ability to communicate
with both business and technical teams.