Jump to content

[API] Historical Data Allowance


Recommended Posts

I programmed a bot to automatically create and close orders for me based on several moving averages and trade momentum. I've tried to find a broker with an API for quiet a while now and I think IG might be the best choice for me, but I'm really having trouble testing my trade strategies mainly due to the low data point allowance on historical data.

The advantage of using a bot is partially its high ability for number-crunshing, but the incredibly low limit of 10,000 data points per week (~60 points per hour) is just not enough. My bot is iterating through the markets, checking for positions with positive trends. Once it finds them, I want to backtest my strategy for this position to see if the bot would have made or lost money with it. The resolution I'd need for that would be in hours, but for just one year worth of data that's 8,760 data points already and then I'd have to wait a week to get more. It's just not feasible.

I know that IG probably has these data limits to prevent people from abusing the API, but the main culprit would not be the amount of data points, but the amount of request. Is there a specific reason why historic data requests are not handled like any other request going to the API (Limited to 60 requests per minute) and instead has this weird way of handling it with data points? The current amount of data could easily be interpreted by a human looking at a chart, which deafeats the whole purpose of having a bot.

My suggestion would be to simply set the allowed amount of requests for historical data to 60 per minute, if that limit is not applied already anyway. Set the amount of data points per page to 250 (or 100, or 500, whatever you feel comfortable with) instead of letting the user decide and the just return a link to the next page like it's already being done. This way we could comfortably get all the data we need while still being throttled.

It would be great if you could consider it.

Link to comment

Maybe you can apply for a corporate account? Not sure if they charge you then though.

Email: newaccounts.uk@ig.com

Other option/workaround (although not sure if IG wants to see this :D) : Work with multiple API keys. Once one has reached its limit, swap over to the next one. Rotate like 5 keys through, at least that way you have 50,000 historical data points

Or: Look if you can optimise your query. Sometimes it's better to scan on a broad level, e.g. daily or monthly timeframe, then we you find something interesting, zoom in to a hourly or 5-minute chart or whatever it is you need for your strategy.

 

In regards to backtesting, try the backtest functionality of ProRealTime. I think it's pretty powerful. No real need to use the API imho.

Link to comment
36 minutes ago, DSchenk said:

Maybe you can apply for a corporate account? Not sure if they charge you then though.

Would a corporation account remove that limit? I imagine I'd actually need a corporation to be able to sign up for one?

36 minutes ago, DSchenk said:

Other option/workaround (although not sure if IG wants to see this :D) : Work with multiple API keys.

I do know that this is not allowed for real-time subscriptions for the lightstreamer interface, not sure about the REST API. Even with 5 accounts though, 50.000 data points per week are still rather low. It's just pretty outdated considering today's technology, but on the other hand, IG is one of very few European brokers who even have an API. It just feels a bit abandoned, because requests to raise the historical data limits are made since 2014 and my IG-Labs account is still not activated after a week, hence me giving feedback here. 😃

36 minutes ago, DSchenk said:

Look if you can optimise your query.

I'm already querying data for positions only once and then store them in my own database, so once I have the necessary historical data for one position, I'm pretty much set for it. My problem are the 40 other positions and I don't want to wait 40 weeks. A hourly resolution is the minimum I need for my strategy unfortunately. i could maybe go with 6 months for backtesting, but I'd prefer a year since I'd like to hold positions over a long period.

36 minutes ago, DSchenk said:

In regards to backtesting, try the backtest functionality of ProRealTime. I think it's pretty powerful. No real need to use the API imho.

I'm using the API to be able to completely automize the lookup of trades, strategy testing and trading. The goal would be for me to not have to do anything at all. 😃 Not sure how I would include ProRealTime into that workflow, unless they also have an API I could use for backtesting. I just need to be able to connect to stuff somehow.

Edited by Nexuscrawler
Link to comment
2 hours ago, Nexuscrawler said:

I programmed a bot to automatically create and close orders for me based on several moving averages and trade momentum. I've tried to find a broker with an API for quiet a while now and I think IG might be the best choice for me, but I'm really having trouble testing my trade strategies mainly due to the low data point allowance on historical data.

Hi, this question has come up a number of times in the past, I don't use the API but have you searched the forum for previous threads on the subject? If you get nothing on the forum search box use google with 'api data allowance' and add 'IG community'.

image.png.1f15e39ebf81733b11554404d6c2d2ab.png

  • Like 1
Link to comment
4 hours ago, Caseynotes said:

Hi, this question has come up a number of times in the past, I don't use the API but have you searched the forum for previous threads on the subject?

I did, yes, but most of these questions are about the error message you get when you exceed your data allowance, and not actually asking to raise it. The most recent question about it I could find was asked in 2014 and ended with IG saying that they'll look into it. Since it's been more than 5 years I wanted to bring it up again, because it's currently the biggest limitation you have when developing automatic trade programs.

Link to comment
1 minute ago, Nexuscrawler said:

I did, yes, but most of these questions are about the error message you get when you exceed your data allowance, and not actually asking to raise it. The most recent question about it I could find was asked in 2014 and ended with IG saying that they'll look into it. Since it's been more than 5 years I wanted to bring it up again, because it's currently the biggest limitation you have when developing automatic trade programs.

ok, I vaguely remember peeps discussing it here in the last few years and thought someone had a get round for the problem but I may be wrong, anyway there are a few api'ers check in on this forum so maybe over the weekend someone might offer up some news. Reply to this post tomorrow sometime to keep the thread up on the noticeboard so others might see it and respond.

Link to comment
  • 6 months later...

If you email them they will increase your allowance temporarily. I was using a demo account too, they might be more friendly if they can see trading/income potential.

I used to hit the limit regularly, but now I keep a database of prices, and I only need to update these daily. So I am using much less data.

edit: What assets are you looking at? Can you get data from another source?

Regarding the IG API in general, its not the best. I really like the web interface, and I used PRT for a while and liked it. But I do have a Oanda account purely because their API is much nicer!

 

Edited by Bopperz
Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • image.png

  • Posts

    • Что такое реферальный код Gate.io? Gate.io определяет реферальный код как специальный код, который новые пользователи могут использовать при регистрации на платформе. Благодаря коду «gate-SEBAMEYA» новые участники могут совершать транзакции со ставкой комиссии, определенной при создании учетной записи. Справочный код Gate.io: Gate-SEBAMEYA В настоящее время реферальный код Gate.io используется как «gate-SEBAMEYA». Это наиболее часто используемый ссылочный код Gate.io в мире, действительный пожизненно, с самой высокой ставкой скидки 70%.
    • Mã mời giới thiệu Mexc là gì? Mexc định nghĩa mã mời là một mã đặc biệt mà người dùng mới có thể sử dụng khi đăng ký trên nền tảng. Nhờ mã "mexc-1925", các thành viên mới được phép giao dịch với tỷ lệ hoa hồng được chỉ định khi tạo tài khoản. Mã giới thiệu Mexc: mexc-1925 Hiện tại, Mã giới thiệu Mexc được sử dụng là “mexc-1925”. Đây là mã giới thiệu Mexc được sử dụng nhiều nhất trên thế giới, có giá trị trọn đời, với tỷ lệ chiết khấu cao nhất là 70%. Khi người dùng đăng ký bằng mã giới thiệu Mexc, các mã này sẽ cung cấp một khoản chiết khấu hoa hồng nhất định, giúp thực hiện các giao dịch tiết kiệm hơn trên nền tảng Mexc. Ngoài ra, tiền thưởng kiếm được thông qua hệ thống giới thiệu có thể được sử dụng trong các dịch vụ khác nhau do Mexc cung cấp, do đó tạo ra môi trường đầu tư hấp dẫn hơn cho người dùng. Khi tạo tài khoản Mexc, hãy nhập mã mexc-1925 trong đó có nội dung “mã mời” (tùy chọn). Mức chiết khấu hoa hồng bạn sẽ nhận được là 70%. Bạn cũng có thể nhận được phần thưởng đăng ký 600 USDT bằng cách hoàn thành các nhiệm vụ trong trung tâm phần thưởng.
    • 什么是Mexc推荐邀请码? Mexc将邀请码定义为新用户在平台上注册时可以使用的特殊代码。借助“mexc-1925”代码,新会员可以在创建账户时以指定的佣金率进行交易。 Mexc推荐码:mexc-1925 目前,Mexc推荐码使用“mexc-1925”。它是世界上使用最多的Mexc推荐码,终身有效,最高折扣率为70%。 当用户使用Mexc推荐码注册时,这些代码提供一定数额的佣金折扣,从而可以在Mexc平台上进行更经济的交易。此外,通过推荐系统获得的奖金可用于Mexc提供的各种服务,从而为用户创造更具吸引力的投资环境。 创建Mexc账户时,在“邀请码”处输入代码mexc-1925(可选)。您将获得的佣金折扣为70%。您还可以通过完成奖励中心的任务获得600 USDT的注册奖金。
×
×
  • Create New...
us