AnsweredAssumed Answered

Handling incomplete data in response

Question asked by Christopher Smith on Jul 24, 2018
Latest reply on Jul 27, 2018 by Christopher Smith

Hello all,



I have a python script that has been running daily since about this time last year. Only in the last few weeks has this issue seemed to crop up. I'm not sure if it is on Qualys' backend or if something else is to blame. I'm utilizing the requests library and a session-based login. When downloading data from API v2, more often than not I will receive a response that is quite literally cut off. For instance:




   <VULN_TYPE>Information Gathered</VULN_TYPE>



   <![CDATA[DNS Host Name]]>


I have experienced this issue with both Host List VM Detections and fetching scan results. I have handlers for HTTP errors and connection reset errors, but this does not seem to be the issue. Where the data cuts off is inconsistent as well. Sometimes it's in the middle of a CDATA section, sometimes it's in the middle of a tag. Either way, there is no indication of any connection problems. It seems to be a complete crapshoot as to whether or not this will occur. Some days there are no problems whatsoever. Other days I can't get anything downloaded without trouble. For now, I have set up a try/catch that will toss out whatever data was being pulled and repeat the request until receiving a complete response that can be parsed. 


Has anyone else dealt with this sort of issue, and if so, how did you resolve it?