The Data Services group is responsible for core data and analytics services which provides a single point of entry for Bloomberg Terminal and Enterprise applications to retrieve any kind of data available in Bloomberg. This includes multiple foundational services such as Data Gateway, Security ID Master, Metadata Infrastructure, Insights and Anomaly Detection System. These systems serve several hundred billion requests a day running on thousands of machines. With ever increasing data volumes and growth of Bloomberg's Enterprise products, this footprint has been growing at a rapid pace (increasing 25% YoY).
What's in it for you:
Re-architect the next generation of our infrastructure
Design, implement and own critical applications and components of our infrastructure stack
Work on high visibility projects which have outsize impact with many opportunities to interact with Senior Management.
Incorporate open source and industry standard solutions to solve the problem at hand
Interact with various teams across Bloomberg to evangelize your work and help people in adopting your system
You'll need to have:
4+ years working experience
Knowledge of object-oriented design, data structures, algorithms, and all phases of software development. This experience can be in any object-oriented language, but open to working with Java, C++, Python, or React
Experience developing large scale Microservices systems
Experience with high volume, high availability distributed systems
A Degree in Computer Science, Engineering, Mathematics, similar field of study or equivalent work experience
Read on to learn more about our teams and what we're looking for: You can express interest in particular teams during the interview process and we'll do our best to match you with your preference! If you are not sure, we will work with you to try to find a role that matches your interests and skills.
Data Gateway Data Gateway provides a single point of entry for Bloomberg Terminal and Enterprise applications to retrieve any kind of data available in Bloomberg. It is the backbone of the Bloomberg application stack, is very widely used and serves 400+ billion requests a day.
To accommodate our future needs we are working on re-architecting large parts of our system. We envision the system to follow an API Gateway pattern with hundreds of data provider microservices integrating into it and providing data for their bounded context. Data Gateway absorbs the complexity of multiple microservices with their myriad details and orchestrates requests to them as needed, thereby providing a clean interface to the outside world and decoupling the services from their clients. To ease the access and manageability of this infrastructure, we are working towards 'Data Gateway as a managed service' that would run on a large farm of shared machines guaranteeing SLAs, providing key functionalities such as routing requests to appropriate services, traffic isolation, authorization and authentication, caching, fairness, load balancing, request shaping, transparency and tools to clients to manage their tenancy. It will ensure that the system is scalable to handle billions of requests from thousands of applications. The biggest challenge we face is to design the system to be resilient to unexpected client load, system failures and latency spikes in downstream services and to auto-isolate such failure while still being able to serve the client requests. You'll work with C++
Anomaly Detection and Insights The system catering 400+ Billion data points poses unique large scale distributed system performance, data quality and Insights problems and requires an early Anomaly Detection system and Insights platform. In order to provide highly available, consistent and correct data in a timely fashion, we need infrastructure to detect the performance and data anomalies. The infrastructure needs to be running at petabyte scale. Anomaly Detection team has successfully built a performance anomaly detection system using spark on Hadoop. Over years we have pruned it to reduce false positives to an extent that we run at more than 90% efficiency. Data Anomaly detection poses bigger challenges because the data sources and types are numerous. With petabytes of data and thousands of microservices providing this data, the problem becomes ever more difficult to solve. We need to build smart, low latency, scalable solutions with good visualization tools to slice and dice the issues into actionable categories.
To provide Business Intelligence & Analytics for Data Gateway usage and empower Product & Engineering teams across the firm to make data-driven decisions we have the Insights platform. We are building out infrastructure that supports a self-service model to onboard application teams to access mission-critical data, and provides a scalable way to help teams gain data-driven insights into how their applications function based on terabytes of data. The jobs will use Apache Spark to scrape and parse data, which will then be stored in an Apache Hive cluster where the data will be accessible to any client who requests it.
You'll work with: Python, Hadoop, HDFS, HBase, Spark, Tensorflow
Bloomberg is an equal opportunity employer, and we value diversity at our company. We do not discriminate on the basis of age, ancestry, color, gender identity or expression, genetic predisposition or carrier status, marital status, national or ethnic origin, race, religion or belief, sex, sexual orientation, sexual and other reproductive health decisions, parental or caring status, physical or mental disability, pregnancy or maternity/parental leave, protected veteran status, status as a victim of domestic violence, or any other classification protected by applicable law.
Bloomberg is a disability inclusive employer. Please let us know if you require any reasonable adjustments to be made for the recruitment process. If you would prefer to discuss this confidentially, please email email@example.com.