Identifying and Removing Duplicate Unique IDs (UUIDs) in Qualys
Duplicate UUIDs normally appear when an asset is “cloned” that already contains a Qualys UUID. The UUID is stored in the registry key HKLM\SOFTWARE\Qualys\HostID on Windows systems and in the /etc/qualys/hostid file on Linux servers. Cloning is a great time saver, so just remove the UUID before or after the cloning process to prevent duplicate UUIDs.
Signs there a duplicate UUIDs in your network
1. If you scan a number of assets in your environment and you can see they were successfully scanned in the raw scan report, but when you try to find the asset in asset search, the asset cannot be found. This happens because Qualys combines the raw scan results, so two assets with the same UUID become one asset, thus you can find one but not the other.
2. You have a number of unused authentication records that no matter what you do they don’t get used. You may also see a flip flopping of unused authentication records, i.e. you scan asset IP 126.96.36.199 and the authentication record for 188.8.131.52 becomes unused. If you scan IP 184.108.40.206 the authentication record for IP 220.127.116.11 becomes unused.
Ways to find duplicate UUIDs in your network – small number of IPs
1. View a raw Qualys scan report and copy the “Successfully Scanned Hosts” IP address list from within the
appendix section, i.e. 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206 and paste it into the asset search IP ranges box and
press the search button. If for example 220.127.116.11 does not appear in the search results, there’s a good chance it has a duplicate UUID, assuming fully authenticated scans are working properly.
With a small number of IPs like this example you can remote into 18.104.22.168 and obtain the UUID from the registry setting HKLM\SOFTWARE\Qualys\HostID or the file /etc/qualys/hostid and record it. Then perform an asset search for all asset groups searching for QID 45179 with matching results for the host ID you obtained, i.e. 11111111-2222-3333-4444-555555555555.
If another asset is returned, i.e. 22.214.171.124, with that host ID you’ve found your duplicate ID. Remote back into
126.96.36.199 and remove the hostid. Because all of the vulnerabilities of 188.8.131.52 and 184.108.40.206 have been combined into
220.127.116.11, you’ll want to purge asset 18.104.22.168 from Qualys and rescan both 22.214.171.124 and 126.96.36.199 as soon as possible.
When scanning has completed, do an asset search to make sure both 188.8.131.52 and 184.108.40.206 are found and their
authentication records have been used.
Ways to find duplicate UUIDs in your network – large number of IPs
1. If you have a tool, i.e. BigFix, you may be able to create a report that will tell you the UUID of each asset. You can then export the report as a CSV, import it into Excel, and highlight assets with the same UUID.
2. Without such a tool, you can find the information using Qualys, but it is a lot of work.
a. Download the raw scan reports for each of your scan groups, i.e. main location, remote offices, and disaster recovery site, as CSV files.
b. Place each of the CSV files into the same empty folder, i.e. c:\Qualys.
c. Combine the reports into one CSV file by going to the Windows command prompt, changing to the appropriate directory, i.e. c:\Qualys, and typing copy *.csv combined.csv.
d. Open the combined.csv file in Excel. If the file is too large, the 32-bit version of Excel may not be able to open it, so you may need to work with each individual CSV file.
e. Rename the file to something more meaningful, i.e. Qualys – Combined Assets – mm/dd/yyyy.csv
f. Save it in the XLSX format. This step will reduce the file size and Excel seems to perform better.
g. Remove unnecessary header information and columns. Be sure to keep columns IP, DNS, NetBIOS, OS, QID, and Results. Also remove any odd IP addresses at the bottom of the sheet. This step will also reduce the file size and allow Excel to perform better.
h. Save the XLSX file before proceeding.
i. Filter the sheet by QID 45179.
j. Ctrl + A to select all and then cut and paste the values into a new Excel tab/sheet.
k. On the new tab, sort the sheet by the results column.
l. Highlight the results column and go to Home > Conditional Formatting > Highlight Cells Rules > Duplicate Values… >
n. The results will look something like this.
o. Purge the duplicate assets from Qualys.
p. Remove the UUIDs from the assets in either the registry at HKLM\SOFTWARE\Qualys\HostID or the /etc/qualys/hostid file.
q. Rescan the affected assets at the earliest opportunity.
r. When scanning has completed, do an asset search to make sure all of the rescanned assets are found and their authentication records have been used.