Our company is seeking a skilled and experienced Database expert to help us understand the ins and outs of Elastic Search and Apache Spark functionality. The ideal candidate should have advanced knowledge of these systems and extensive experience working with large databases (from TBs to PBs). Additionally, the candidate should possess an understanding of the algorithms running under the hood of ElasticSearch and/or Apache Spark. Having experience in performance profiling and tuning specific use cases would be a huge plus. It would also be a significant advantage if the candidate has contributed to the development/customization of Elastic Search and/or Apache Spark.
Ideally, we are looking for a candidate who meets all of our needs, but please apply if you believe you are capable of fulfilling some of our requirements.
Responsibilities:
-
Consult the engineering team about typical use scenarios and performance bottlenecks of ElasticSearch and/or Apache Spark.
-
Create realistic performance benchmarks for those systems so that the hardware team can use them to compare the current implementation with a hardware-accelerated one.
-
Be open to modifying the internals of those systems in order to interface with hardware accelerators (drivers and C APIs to hardware accelerators will be supplied).
Skills:
-
Degree in computer science, engineering, or a related field
-
Minimum of 5 years of relevant experience
-
Strong experience in Non-relational databases
-
Strong understanding of database architecture
-
Expertise in Java programming and profiling (for Elastic Search/Lucene)
-
Knowledge of Scala programming and profiling (for Spark)
-
Knowledge of Elastic Search/Lucene internals is a plus.
-
Knowledge of Spark internals is a plus