Skip to main content
main content, press tab to continue
Article | Beyond Data

The hunt for competitive compensation data

Finding the right data source for your organization

By Sambhav Rakyan and Hatti Johansson | August 31, 2023

Web-scraped. Crowdsourced. Generative AI-based. Options in the search for competitive pay data are increasing. What’s the right answer?
Compensation Strategy & Design|Ukupne nagrade
Beyond Data

The world of work and rewards is changing. Compensation and HR professionals are juggling a rapidly changing economic landscape, rising labor costs, increased competition for talent and volatile economic pressures. All of this is compounded by changing legislative and regulatory policies aimed at protecting data privacy and ensuring pay transparency and pay equity.

These rapid changes are raising questions about the pay data upon which decisions are being made. There is an increasing sense that organization-trusted validated data that is even just a few months old may already be out of date. Now add the context of a data-intensive world and organization leaders who expect compensation and HR professionals to have all the answers to all the questions right now, while at the same time responding to the various data points and sources that are coming at them.

In responding to this laundry list of challenges, compensation and HR professionals need to be confident that they have the best information for talent acquisition, engagement and retention discussions. They also need to be prepared to respond to true vs. anecdotal salary data points that employees raise.

The reality is, not all data sources are equal, and that’s true regardless of when or how quickly salary data is delivered. Additionally, not all external influences are felt the same way in all regions, industries or company sizes. Identifying the salary data that really matters requires a closer look at the factors that influence strategic business decisions.

Annual survey data is still the most effective tool

The economic volatility of the past few years has been the platform to reiterate and communicate that salaries are not, in fact, responsive to labor market changes. With a strong propensity to control fixed costs, compensation and HR professionals look to tightly manage salary budgets.

Even with labor market shortages and inflationary pressures – which workers have leveraged to demand higher pay and better benefits – employers are cautious about being the first to significantly raise salary budgets. Changes may be made, but only after careful consideration rather than as a knee-jerk reaction. And it’s important to remember that change can take many forms: incentives, benefits and even an enhanced purpose-drive employee experience.

Annual compensation surveys continue to provide the most accurate and trustworthy data for a variety of reasons:

  1. You know you can trust the data source. The best salary surveys are built around data that are collected from companies and submitted by compensation professionals. They also are based on a consistent and transparent survey job matching methodology and are validated by compensation experts.
  2. You can review the quality and reliability of the results. Salary surveys report the organizations in the sample, including how many organizations and incumbents are represented in the data (e.g., by base salary, bonuses, allowances) so you can tailor the results to ensure you have the closest fit to your organization and understand where your talent is coming from and going to – all while being compliant from an anti-trust perspective and meeting all legal requirements.
  3. You can ensure the results are regularly updated throughout your compensation planning period. We know that pay review periods aren’t a one-size-fits-all. Yes, 70% of participants in WTW surveys follow a calendar-year pay-review schedule, but what about the remaining 30% in our surveys? They submit their data through year-end, effectively refreshing our database with additional data points as frequently as daily.
  4. You can use data from compensation surveys in combination with other trusted data sources for a complete picture.

Additionally, most organizations review their compensation data – and deliver merit and cost-of-living increases – annually. In fact, we know that more than 85% of organizations conduct their reviews in the first six months of the calendar year. Additionally, the results of our global compensation surveys tell us that most compensation-related decisions remain valid until the next annual review. Only a small proportion of data points tend to change, and those changes are attributed to net new hires or leavers. In most organizations, that’s a small percentage of their overall headcount.

Of course, organizations may need to adjust their compensation plans based on extraordinary marketplace factors. For data questions that need a timelier response, pulse surveys on a specific topic and market intelligence can reflect more recent information. For example, WTW's Salary Budget Planning Report is conducted bi-annually, allowing for a midyear understanding of how organizations may be adjusting their compensation plans as well as a view into final decisions that may be made by year’s end.

One final note about annual compensation reviews: Most salaries change only once per year (unless someone starts a new job or receives an off-cycle adjustment). Complicating matters, many companies make their salary adjustments in different months. As such, real-time data may only capture fluctuations based on the organizations that most recently provided data. This results in an inaccurate picture that market data is higher or lower at any given moment rather than acknowledging the broad market reality for a job.

Strategic decisions depend on reliable market information

Google has made it easy for anyone to explore typical pay for their role – and then turn around with their research in-hand to question the competitiveness of their salary. However, experienced compensation and HR professionals understand the proper role for these online data sources. Whether it comes from web-scraped job postings, crowdsourced information or even generative AI, these are all indicators or reference points when no other data is available.

Web-scraped data

Data that comes from job-post information is extracted from job postings or recruitment websites. Typically, the data includes the:

  • Job title
  • Job description
  • Job location
  • Advertised salary range
  • Company information

This information can be used to compare job posts across specific geographic locations, identify labor-demand trends over time and even provide transparency about which organizations are recruiting for certain roles.

Web-scraped salary information may feel more real-time, as it can be based on the most recent crop of job posts; however, it is impossible to know if, when or how the jobs are filled. Also, web-scraped data is an incomplete picture of current market rates, as it only reflects advertised salaries or salary ranges rather than the actual rate of pay for a particular job.

Crowdsourced pay data

This self-reported data is collected from individuals who voluntarily submit their earnings information in exchange for a market rate for that job. Potentially useful as a cost-effective reference point or to compare salaries across geographies when other information is unavailable, it is important to remember that crowdsourced data reflects unvalidated results that may change frequently based on who from “the crowd” provides their salary information. It can be challenging to know the source or reliability of the data, and the information may be a mix of annually updated salaries and new-job holders’ information.

Again, crowdsourced pay data may feel more current because results may seem to be published more frequently, but it is difficult to know when the incumbent’s salary was last updated or how “fresh” their self-reported information may be. In short, crowdsourced data lacks the nuance and sophistication of tested, validated and credible employer-verified data.

Generative AI salary data

Created by AI algorithms, this data can contain estimates of salary ranges for various job positions and economic areas as well as other salary-related information. These data sets can be generated for quick reference purposes or to help inform informal salary negotiations, but generative AI salary data is the least reliable source of market data – it is based on a model or algorithm that lacks transparency into the input factors. It also may lack important factors that affect pay such as skills, work experience and level of expertise.

When considering each of these alternative data sources, it’s important to remember that most organizations – regardless of whether they formally articulate it – have pay philosophies that focus on attracting and retaining talent by paying competitively relative to a targeted, competitive labor market. And it is important to use the most reliable, accurate and trustworthy data to make critical and defensible pay decisions.

When more up-to-date data is needed

While the annual compensation survey provides accurate data for many organizations and roles, about one-third of participants in a WTW client feedback survey said they need more frequent updates for at least some select in-demand jobs. In response, niche data is beginning to enter the survey market to focus on what’s trending, allowing organizations to respond before demand overtakes supply.

For example, WTW’s Hot and High-Demand Jobs Report – U.S. leverages historical data to identify how hot a job is across multiple dimensions, including recent and sustained pay increases, surges in demand and top paying disciplines. These dimensions can identify which jobs are more in-demand at selected career levels and support pay strategies for attracting and retaining these roles.

Hot jobs show the importance of understanding the impact of skills on pay as that can help attract and retain critical talent. Accessing the right trends and skills prevalence data supports defensible compensation decisions and broader organizational planning.

Another instance in which salary may change more frequently is in geographies that are hit particularly hard by hyperinflation, like Turkey and Argentina. In these cases, pulse surveys can monitor how unstable market conditions, underdeveloped HR functions and wide skills gaps are affecting approaches to rewards.

The best of both worlds: Data and best practices

The world isn’t slowing down, so having the best, most trustworthy data and insights is important when making decisions about your organization’s largest investment: your people. Data and intelligence ensure compensation and HR professionals are confident in their recommendations and know their decisions are defensible at every level of the organization.

There are a lot of assumptions and misconceptions about the value of real-time data, its availability and how it can be used. This makes it important to be able to educate organizational leadership about the true value of comprehensive, tested, reliable data as well as the factors that influence strategic – not just-in-time – compensation decisions.

The reality is, there is no one-size-fits-all approach. Finding the best rewards recipe for your organization requires the right combination of ingredients. It also is important to have a strong partner who helps you respond to senior leadership questions around data currency, accuracy and efficacy. That winning combination will build outstanding compensation and total rewards programs that position your organization to find and keep the talent you need.


Global Innovation and Product Development Leader, Rewards Data Intelligence

Global Business Leader, Rewards Data Intelligence

Contact us