Lightstreamer Connect fail on DEMO account

[swingwin]

I can't connect to the DEMO stream ("https://demo-apd.marketdatasystems.com") since December 2021 with a C# .NET application using the IG API
On the other hand the connection to a LIVE stream ("https://apd.marketdatasystems.com") works

It seems that the problem comes from the "DotNetClient_N2.dll" library.
Has anyone encountered the same problem ?

 

Edited January 22, 2022 by swingwin

[teebeast]

I fixed this issue in my Fork and reworked some things that where outdated. Look at this Issue #12 and Issue #18  for more information about the changes in my fork.

I asked if I should pull request my work, but i got no answer.

[swingwin]

@(teebeast) :

Thanks for your help.
I tried with your Fork but here is the message I get:

 

[Tartempion]

Hello Swingwin, I have the same problem with my own LabVIEW software which does not use the dotnet library from IG. After reading your message I tried my software on the DEMO stream and I see the same problem as you. (I rarely use the demo stream). Do you know what needs to be changed very concretely in the subscription procedure? I don’t program with C#. Everything is at the socket. Did you read somewhere a message from IG that informed about the modification of the parameters for the DEMO stream subscription requests? It scares me if one day they do the same with the PROD stream ahah. Anyway, it's nice to meet you again. We should stay in touch!

[swingwin]

Hello Tartempion, thanks for your answer and for your tests
It's also a pleasure to see you again.

I asked the support of IG France about this problem, but after consulting their technical service, they tell me that nothing has been changed on their side to access the DEMO stream.
The problem still exists despite the advice provided by Teebeast.
But I realize that the problem does not exist when using the IGLabs companion to access the stream.
The java example provided in IGLabs also works.
This problem is very disturbing.
Especially since I haven't changed anything in my apps for months, and everything was working fine until mid-December 2021.  

 

 

[Funky_Gibbon]

btw, I'm in issue #12 as Slashernator

[Funky_Gibbon]

Hi, I have this problem too.
This is the stack trace.  Looks like the backend has changed.

[jlz]

Hi, the .net example is far from ideal. I ended developing my own library. An example of how to use the LightStreamer client with it is at:

https://github.com/AnAlgoTrader/IG.Csharp.Api.Client/issues/2

It does not use the DotNetClient_N2.dll where the main problem seems to be at.

Edited February 4, 2022 by jlz

[Funky_Gibbon]

Hi, thanks for this.  Have you tried against the demo lightstreamer subscription?
Just asking as my prod one is fine, we're just finding that the demo one fails.

Cheers😊

[jlz]

Hi, it does work fine.

I left the example at:

https://github.com/AnAlgoTrader/IG.Csharp.Api.Client/blob/master/Playground/Program.cs

Edited February 5, 2022 by jlz

[Funky_Gibbon]

Excellent.  Thanks for trying.  I'm going to alter my trading algo to use this library.
Wondered if the old one uses .net 2.1 which is now out of support?
 

[Funky_Gibbon]

Hi, Unfortunately my project uses .Net framework 4.5 and I can't attach this library as the error is telling me they're incompatible.
I've tried all sorts but not strong enough to understand the issue other than rewriting the using statements?

Any ideas?

 

[jlz]

I have changed the type to .net standard 2.0. That should let you load it with both 4.5 and .net core. Pull the latest and rebuild the project.

If you have any other issues with the library please log an issue on Github, I don't get notifications from this forum if I don't log in. Github will send me an email with your issue.

[TommyT]

Hi Swingwin (and other contributors),

I too had a similar problem with the DEMO stream, however my timings were different. I had no issues in December but started to get the same issues as you in early Jan. At the same time as I had connection issues with DEMO, the streaming companion on demo worked fine, and the same code DotNetClient_N2 connect to LIVE stream worked fine

There are other posts on the forum where others state it has happened to them, but during different periods. Given it seems to affect different people at different times, and those people seem to be using it then it suddenly stops for a while, I am beginning to wonder if this is down to the stated data limit.

“40 concurrent subscriptions”.

I wonder if, when an application has the stream up and is killed during testing/debugging, does it leave the subscription count incremented? This would seem to explain why different people get it at different times. If then at some point the count is reset access comes back – which it seems to. And this would also explain why it mostly happens with DEMO. (I have had it happen on LIVE too when I have had to use LIVE when DEMO was not available)

A natural counter to this would be that if this is the case it should block the companion connection too. However, I used F12 dev tool on the browser to look at the mechanism for connecting the companion.

From what I can tell there is a slight difference in the login request headers in the PUT to /gateway/deal/session

The c# code I have adds X-IG-API-KEY and VERSION as default headers, and Content-Type as custom to that request particular, whereas the companion uses

      req.headers = {

            "X-IG-API-KEY": credentials.apiKey,

            "Content-Type": "application/json; charset=UTF-8",

            "Accept": "application/json; charset=UTF-8"

        }; 

Not sure what “Accept” is doing.

Regards Tom

[Funky_Gibbon]

Hi Tom,

Thanks for this, very detailed.  I've just tried lots of GO's on the streaming companion, actually 5 and had the error message.
"Login failed: 403 (Forbidden) error.public-api.exceeded-api-key-allowance"

So likely this is the cause.  Still not sure how a concurrent subscription happens with only one link.  I'll send this to IG.

[Funky_Gibbon]

Funnily enough my demo account is working.  I'm going to try and use the new library I'm not really a coder although I roughly know what I'm doing and its a little different but here goes

[jlz]

7 hours ago, TommyT said:

 

Not sure what “Accept” is doing.

 

The Accept header is just to force the server to send you a specific format, a json format in this case.

It is sent as a header because many servers would use xmls as the standard if nothing is specified.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept

 

Edited February 8, 2022 by jlz

[TommyT]

On 08/02/2022 at 12:48, Funky_Gibbon said:

Hi Tom,

Thanks for this, very detailed.  I've just tried lots of GO's on the streaming companion, actually 5 and had the error message.
"Login failed: 403 (Forbidden) error.public-api.exceeded-api-key-allowance"

So likely this is the cause.  Still not sure how a concurrent subscription happens with only one link.  I'll send this to IG.

Hi Funky,

I was using the LS dot net Client dll. it supports calls to "connect" and "disconnect" which is pretty opaque. I did not/could not find the level of detail you found (error.public-api.exceeded-api-key-allowance) . It seems to confirm it, though am not sure how the streaming companion escapes this limit even though it uses the same API key and account

regards Tom 

[Funky_Gibbon]

Hi Tom,
Using the LS dot net client, I couldn't retrieve a detailed error.  Only using the streaming companion gave this info.  However, I'm converting to use the other method from Algo Trader.

Cheers, Ad

[swingwin]

As the Teebeast fork does not fix the problem for me,
As I didn't test the Jlz solution,
As the java example works in LIVE and in DEMO,
As the IGlabs companion works LIVE and in DEMO,
As my application (few thousands lines of code) use C#.NET,
I will interface the JAVA lightstreamer library with C#.NET using the IKVM.NET tool.
I will specify here as soon as it works and fixes the problem.

[Funky_Gibbon]

Just to let you all know that my DEMO account was working last week and now it's not again.

[swingwin]

I have not made any progress with IKVM since 17 February.

With the LightStreamer .NET library, always the same problem: connection in LIVE OK but connection in DEMO impossible.

I tried @jlz solution, but same problem

[teebeast]

Which version of LightStreamer .NET library do you use? I tested the NuGet-Package Lightstreamer.DotNetStandard.Client version 5.0.5 and I had no problems with my demo account. Do not use any newer version because its not compatible with IG.

[Funky_Gibbon]

I just rewrote my entire program to use IG.Csharp.Api.Client in WPF forms.  The whole thing is more stable than before with messages that show what's happening with the lightstreamer client.
Fascinating how it pauses from time to time. 

[charlie303]

Hi,

I'm also having problems connecting to LightStreamer DEMO. My code which used to work fine has recently stopped working. Everything is fine for PROD. Even the undocumented 'Heartbeat' function works. This is using the .Net client  DotNetClient_N2.dll.

I tried jlz / algoTrader's code (thanks) and got that working for PROD but again not for DEMO. This code uses the .Net Standard LightStreamer client v5.0.5 as I understand it.

So that's 2 different C# libraries not working for DEMO.

The IG Companion works fine, not had a single problem with that.

I've not yet had the time to delve into the HTTP headers, I used to use SOAP UI for that kind of deep dive so I don't know what error codes are being returned if any. Someone here posted about -

"Login failed: 403 (Forbidden) error.public-api.exceeded-api-key-allowance"

which seems plausible but I haven't diagnosed for so I can't verify. It wouldn't explain to me though why I can still use the IG Streaming Companion concurrently whilst the C# code fails. I could for example connect to LightStreamer DEMO in Brave browser (SUCCESS), then run my code (both client libraries FAIL) and then connect again to LightStreamer DEMO in Oprah browser (SUCCESS). Any permutation on this produces similar results.

It seems to me there is a difference between the PROD and DEMO server settings at IG's end. Is there anyway we can escalate? I really don't fancy developing against PROD or wrapping some kind of Java client.

Thanks for the help on this thread.

Charlie

[teebeast]

Hi Charlie,

try to create a new API-Key for DEMO access. It seems that the Lightstream server has its own blocklist, independant from the REST server. Maybe this is a problem in your implementation. Or is a result of heavy debugging and testing.

[charlie303]

@teebeast

Thanks for the swift reply.

Surprise surprise I'm having difficulty creating a new DEMO API Key. The My IG website says there is a problem so I've contacted IG support and hopefully they can reset something. The fact that I'm having trouble doing this does tend to suggest they have taken steps to ensure their servers aren't overloaded which is what I've read in other community posts and is fair enough.

I'll keep you posted.

[swingwin]

Thanks Charlie for the detailed explanation.
It's exactly the same problems on my side
I also tried to create a new API DEMO key in the past, and had the same failure to create it.
Maybe by doing a deactivation/activation of the existing API key the problem will disappear, but I don't dare to do the manipulation because I'm afraid of a definitive deactivation of this key.

I am very interested in the feedback you will get from IG.

[charlie303]

So I've waited a few days now for a reply from IG (I received an automated one so I know they got my email which included a link to this thread) and so far none has been forthcoming. I'm not surprised if I'm being honest. I've got a feeling that from IG's point of view it is not really worth their time on the issue of DEMO API Keys.

I borrowed a DEMO API Key from a good friend who has had an account with IG for a few years now and had exactly the same problem. I could login from both a browser and from code but the Lightstreamer connection in code failed. I know my friend has not overly used the DEMO API Key for development as he is not a programmer so the problem is not due to overuse.

From reading other posts here on this thread the problem isn't due to using C# libraries, whether using the legacy library (DotNetClient_N2.dll) or the new .Net Standard one (v5.0.5 - which incidentally only seemed to achieve adding 350MB of libraries to my project).
@Tartempion mentioned using LabVIEW at the socket level and having the same problem.
@teebeast mentioned obtaining a new DEMO API Key because of a possible blocklist.

I can only guess that there is some issue at IG's end with DEMO API Keys that were issued sometime ago.
To be fair, I can understand why this is not a priority for IG. The return on investment is probably not worth it for the resources they would have to devote (skilled, knowledgeable personnel, time, money, etc). I guess that the number of retail clients that use the DEMO API and who are financially viable for IG is very, very low. They are probably more invested in their web, app, MetaTrader and ProRealTime platforms.
 
I actually see this as a positive as I believe in my ideas and skills and now speculate there aren't many doing what I am doing. It is a bit of a poor mark though as I feel as an engineer/programmer, you should know your kit inside out and top to bottom as best you can. Or, at least, reply with an email saying it is not IG's priority at this time to resolve the issue.

When I logged onto my friends demo account I noticed that he had trades that were well over a year old and still running (Buy Oil @ US$49!). As a suggestion to IG could they not completely reset/reboot the Demo environment every now and then, after giving advance warning to clients that all demo trades would be wiped?

I have decided to continue development of my software for the IG API. I have over a period of time compiled notes and so I am familiar with the implementation and some of its quirks. For example TRADE CONFIRMS on both the REST API and STREAM APIs isn't ideal. I will have to develop against PRODUCTION using my live account which is far from ideal but there are markets that have low margin requirements and price per points (eg the crypto EOS) so hopefully I won't lose money. I'm glad I am not developing from scratch.
 
I've decided though that in the future any new trading software I develop will be for MetaTrader 4. I only have a limited understanding of its API and I hear debugging is a pain. To many it would look like a step backwards but MetaTrader 4 has one great advantage in that I am not tied to any one broker. So if my account were locked or frozen I may have the option to move elsewhere. This is not intended as a criticism of IG who obviously want active clients but if you look at recent world events in say Canada or Russia, Governments can freeze bank accounts without due legal process or notice.

Onwards and upwards.

Charlie
 

[mf2]

So I had the same issue that my self-developed client simply would not connect to the API anymore; after checking out the LightStreamer API companion (it took some hours to remember that it exists) and examining the requests it makes it seems that the API was upgraded to 6.0, which require a little bit different session creation. But then it works with the demo account just fine, even with an old API key.

[charlie303]

@mf2
Thanks for the reply. I've been going crazy as this weekend (Sat 14 May 2022) my LIVE LightStreamer API Key stopped working, though the API Companion website is still working.

Could you provide me with any more information?
What software did you use to diagnose the API Companion? POSTMAN? SOAPUI? Wireshark?
When you say version 6 - do you mean LightStreamer Server? I have been using from C# both the legacy client (DotNetClient_N2.dll) and the .Net Standard client v5.0.5. Is there a new client? v6?

Any more info would be appreciated. If I get it working I will post a detailed breakdown.

Thanks
Charlie