Using HttpClient in SSIS Script Component with visual studio 2015 framework4.8
-
Hi,
I'm using HttpClient in an SSIS script component to retrieve data from a Rest API JSON, and the issue is I'm getting the same elements in several page even if the parameter page number is changed, the variable result does retrieve only the first ReadAsync... but it seems like if there is a cach that needs to be empty, of course I already added no-cache on the header and datetime.Now, but still having this..
I'm working with visual studio 2015, and framework 4.8
Thanks in advance if by any chance someone has encountered to this before,
-
If the code is not too lengthy, perhaps you could post it here? (Sanitised as necessary.)
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin -
Thanks for your reply, here's my code :
HttpClient client = new HttpClient();
client.BaseAddress = new Uri("https://api.***.***/***/******/**/**");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("*****", "********************************");
string APIUrl = string.Format("https://api.***.***/***/******/**/**");
var response = client.GetAsync(APIUrl).Result;
IEnumerable<string> values;
values = response.Headers.GetValues("X-Total-Count");
int h = 0; int c = 0;
foreach (var headerItem in response.Headers)
{
foreach (var valueItem in values)
{
if (response.Headers.Contains("X-Total-Count"))
{
h = int.Parse(valueItem);
c = (h/1000) + 1;} break; }break;}
for (int k = 1; k <= c; k++)
{
client.DefaultRequestHeaders.CacheControl = new CacheControlHeaderValue { NoCache = true };
client.DefaultRequestHeaders.IfModifiedSince = DateTime.Now;
APIUrl = string.Format("https://api.***.***/***/******/**/**?Page=" + k + "&per_page=1");
response = client.GetAsync(APIUrl).Result;
if (response.IsSuccessStatusCode)
{
var result = response.Content.ReadAsStringAsync().Result;
var serializer = new JavaScriptSerializer();
Appointment[] myDeserializedClass = serializer.Deserialize<Appointment[]>(result);
for (int i = 0; i < myDeserializedClass.Length; i++)
{
Sortie0Buffer.AddRow();
Sortie0Buffer.storeid = myDeserializedClass.store_id;
Sortie0Buffer.storename = myDeserializedClass.store_id;
....etc;}}}}
-
This is a script component source, presumably? (Rather than a transformation.) Is the code part of the CreateNewOutputRows() sub?
Does the version of SSDT you are using allow you to debug script components? I can't remember exactly when that became available. If it does, have you tried stepping through the code line by line to help identify where it's going wrong?
If not, the best advice I have is to create a pure C# application (Windows or Console, whichever you prefer) and port the meat of the code to that. You should be able to recreate most of it and then you will be able to debug where it's all going wrong.
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin -
Yes, I'm using an SSIS script component with SSDT 2015, and I'm able to Debug, when I put a MessageBox.Show(), I can see the link modified with the page parameter k Updated with values (1,2,3, ....) :
APIUrl = string.Format("https://api.***.***/***/******/**/**?Page=" + k + "&per_page=1");
var result = response.Content.ReadAsStringAsync().Result;
But the variable result doesn't change it's content, it remains with the same elements as the frst page...
I'm getting this errors into the debugger when I select Content :
-
Have you verified that the web service call using exactly the same URL as in your code works properly outside of SSIS, using something like Insomnia (https://insomnia.rest/)?
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin -
Yes, I already did the test from CMD and on a dedicated API platform.
I added a screen shot from the debug to the preceding post..
- wiam.elab wrote:
Yes, I already did the test from CMD and on a dedicated API platform.
I added a screen shot from the debug to the preceding post..
Please try again, as I cannot see the screen shot.
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin -
I now added it as attachement, hope you can see it 🙂
-
Yes, I see it. But my C# is not advanced enough to know how to fix that, apologies.
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin
Viewing 10 posts - 1 through 9 (of 9 total)
You must be logged in to reply to this topic. Login to reply