Data mining approach to monitoring the requirements of the job market: A case study

an article by Ioannis Karakatsanis, Wala AlKhader, Armin Alibasic, Mohammad Atif Omar, Zeyar Aung and Wei Lee Woon (Masdar Institute of Science and Technology, Abu Dhabi, United Arab Emirates) and Frank MacCrory (MIT Sloan School of Management, Cambridge, MA, United States) published in Information Systems Volume 65 (April 2017)

Highlights

• A Text-Mining approach for matching raw job advertisement documents with occupation description data in the O*NET database is proposed.
• A Latent Semantic Indexing (LSI) model was utilized along with a wide collection of preprocessed data from online sources.
• Crowdsourcing was deployed to validate the proposed methodology.
• Results reveal that the suggested method can be directly applied to different job markets, commercial sectors and geographical regions.

Abstract

In challenging economic times, the ability to monitor trends and shifts in the job market would be hugely valuable to job-seekers, employers, policy makers and investors. To analyze the job market, researchers are increasingly turning to data science and related techniques which are able to extract underlying patterns from large collections of data.

One database which is of particular relevance in the presence context is O*NET, which is one of the most comprehensive publicly accessible databases of occupational requirements for skills, abilities and knowledge.

However, by itself the information in O*NET is not enough to characterize the distribution of occupations required in a given market or region.

In this paper, we suggest a data mining based approach for identifying the most in-demand occupations in the modern job market.

To achieve this, a Latent Semantic Indexing (LSI) model was developed that is capable of matching job advertisement extracted from the Web with occupation description data in the O*NET database.

The findings of this study demonstrate the general usefulness and applicability of the proposed method for highlighting job trends in different industries and geographical areas, identifying occupational clusters, studying the changes in jobs context over time and for various other research embodiments.

Hazel&rsquo's comment:
I do not wish to come across as a wet blanket but this or something similar has been tried before. Nothing, absolutely nothing, beats the intelligent human when it comes to indexing.

---