Hello,
I'm a Ph.D. student in U.S. I am using ssllabs-scan to download certificates of a bunch of domains.
Recently I found that around 20 mins are needed for scanning a single domain which is too slow for me.
I have roughly 7K domains need to be scanned.
Please advise me on how should I speed up this scanning process.
Many thanks,
Zack
If all you need are the certificates, then SSL Labs is a very inefficient solution. To just download certificates, I'd suggest the openssl s_client command line interface.
/docs/manmaster/man1/openssl-s_client.html
I'd start with something along the lines of this:
echo "" | openssl s_client -host www.qualys.com -port 443 -showcerts
If you want all the grading and reporting offered by SSL Labs, then you must wait for the reports. There are many tests to perform to check for all known vulnerabilities and exploits. The test is deliberately paced to avoid overloading the hosts being tested. Note, tests typically take 1 to 2 minutes per IP address. If a domain name is backed by 10 IP addresses, then 20 minutes to provide the test results is not unusual.
If you are experiencing 20 minutes waits for domain names backed by a single IP address, then please let us know and please share the API responses for the test.
Lastly, when scanning in bulk, please be sure you follow the rates and limits set forth in our documentation.
Hi Keith,
Thanks for your reply, it's really helpful.
I now have a follow-up question. Do you know any other services or tool that can quickly analyze and grade the locally saved certificates, say I downloaded them by following your suggested OpenSSL tool?
In my case, the ultimate goal is to check, or say to grade, whether a certificate is secure or not.
Thanks,
Zack
I agree with Keith on this. Yang, if you are just downloading the certificates and then doing bulk processing on all the certificates obtained you are probably much better off using openssl directly to do this. You could then provide a file of the domains/urls/ips to hit and depending on your throughput you could kick off each OPENSSL command in the back ground to fetch and write the certificate somewhere. You should be able to rip through them pretty quick. Then a second routine to go through each one and pull out what you want from the certificates. I am doing something similar now with just Powershell code.
David
Hi Busby,
Thanks for your reply. It's nice to have such an active forum.
I do have the same follow-up question for you. By any chance, do you know any other services or tool that can quickly analyze and grade the locally saved certificates, say I downloaded them by following your suggested OpenSSL tool?
In my case, the ultimate goal is to check, or say to grade, whether a certificate is secure or not.
Thanks,
Zack
Yang,
I am not currently do that part of it. You might take a look at SSLyze, https://gbhackers.com/fast-and-complete-ssl-scanner-to-find-mis-configurations-affecting-tlsssl-severs-a-detailed-analys… .
I am not doing this one at the moment but it may give you want you need. I think if you spin up a VM of Kali it will already be present so you can try and use it then your scripts will need to call it to generate the data.
May take a little work on your side though. I know another service and you can get an API as a research I believe but I don't think they do grading.
David
I got it and many thanks to you, Busby.
If all you need are the certificates, then SSL Labs is a very inefficient solution. To just download certificates, I'd suggest the openssl s_client command line interface.
/docs/manmaster/man1/openssl-s_client.html
I'd start with something along the lines of this:
If you want all the grading and reporting offered by SSL Labs, then you must wait for the reports. There are many tests to perform to check for all known vulnerabilities and exploits. The test is deliberately paced to avoid overloading the hosts being tested. Note, tests typically take 1 to 2 minutes per IP address. If a domain name is backed by 10 IP addresses, then 20 minutes to provide the test results is not unusual.
If you are experiencing 20 minutes waits for domain names backed by a single IP address, then please let us know and please share the API responses for the test.
Lastly, when scanning in bulk, please be sure you follow the rates and limits set forth in our documentation.
SSL Labs Access Rate and Rate Limiting