As the demand for data increases, innovation expands, and technologies are developed that create opportunity where there was none before.
Yanhong Wang, principal software engineer for Microsoft Azure Data, is someone who jumped headfirst into the new challenge big data creates.
A biologist by training with a PhD in molecular biology, Wang saw an explosion of genomic data through advances in high-throughput bio-technologies while getting her degree. She was drawn to the opportunity and challenge of data management and mining. She ultimately pursued technical computer science skills in addition to her studies, giving her the experience to ultimately double major with master’s degrees in both biology and computer science. An insightful career move.
“The software industry was like a fast train that everybody was trying to catch,” says Wang. “I was not very sure where I would land but decided to jump in anyway.”
A pivot toward data
Not only has the amount of data we produce each year grown exponentially, storage and mining tools continue to advance. Gartner cites by 2022, 75% of all databases will either be deployed or migrated to the cloud. This means constant continual innovation for the tools businesses require to get the most insight from their information.
Wang’s pivot started with a job at database company, Sybase where she, quickly moved from software engineer to senior director of engineering. It was a natural progression from Sybase to SAP, where she was development manager for the HANA column store. Now at Microsoft she is engineering manager for the Azure Data HDInsight Hive and LLAP team.
“Microsoft is on the leading edge of cloud computing and big data analytic solutions. I am proud to be a part of it and the impact it has across industries.” – Yanhong Wang
Working on the world’s computer
Spearheading leading technologies in big data, cloud, distributed computing, artificial intelligence, storage, and open source, the Azure team runs one of the world’s largest big data clusters. This data requires more than 250,000 machines, stores more than 10,000 petabytes of data, and runs tens of thousands of jobs everyday across search, ads, Office, Xbox, and more.
Within the Azure Big Data project portfolio, Wang’s team is working on Interactive Query (also called Apache Hive LLAP for Low Latency Analytical Processing). Interactive Query supports in-memory caching, making Apache Hive queries faster and much more interactive. Customers use Interactive Query to query data stored in Azure Storage and Azure Data Lake Storage in a rapid-fire manner, making it easy for developers and data scientists to work with big data using the business intelligence (BI) tools they prefer.
(Microsoft HDInsight data warehousing scenarios using interactive hive)
Different data career paths
Wang’s managing philosophy is empowerment. If she had not been empowered to change her career path early on, she would be in a much different position. Personal goals and career development are extremely important as the team continues to push into data sets that are increasingly difficult to navigate.
“My biggest goal is to create a culture of inclusion, ensuring everyone is empowered to perform at their highest level,” she says. “Then when individual career goals align with team goals, it drives success from the ground up.”
Whether someone expresses the desire to become a group leader or engineering fellow, Wang makes sure she taps into the network of Microsoft to get them the mentoring they need. She brings in experts to talk, encourages employees to pursue training, and encourages a collaborative environment where everyone shares knowledge and experience. “Satya says it best, in that we are a learning culture with a growth mindset. Our team has to be if we are going to be able to keep innovating.”
Her advice to young engineers? “Commit to a technology based on the long-term impact it will make. Bouncing around each year to the ‘next big thing’ can prevent someone from growing to their full potential and mastering their craft,” she says.
Azure Data in the Bay Area
Wang is on the look-out to hire more engineers – junior to senior level – to help her team reach its goals. As part of Wang’s highly motivated and agile team, new hires will get an opportunity to work with the latest and greatest challenges in the OSS ecosystem and the Hive/LLAP technologies.
If you are interested in a career in data and joining the local team, open roles can be found here: https://aka.ms/MicrosoftBayAreaCareers
(Yanhong Wang and her Azure Data team in the Bay Area)