Jump to content
  • 0

Daily histrorical data - interpretation of UTC (?) date stamp and daylight savings


smx01

Question

Hi everyone,

I am working to download historical daily data via the REST API (crypto EPICs mainly - e.g. CS.D.BCHUSD.CFD.IP). 

I believe the timestamp that comes with the data us UTC. I am finding that for the period where AEST is in daylight savings time (DST), e.g. Oct-6-2019 - Apr-5-2020, the date stamp changes. Here are two corresponding transitions:

Here's what the REST API looks like around Oct-6

Quote

 

Response from IG REST API

{'prices':                         bid                             ask                         spread                  last                         

                       Open    High     Low   Close    Open    High     Low   Close   Open High  Low Close  Open  High   Low Close Volume

DateTime                                                                                                                                 

2019:10:04-00:00:00  221.72  223.35  215.64  222.62  223.72  225.35  217.64  224.62    2.0  2.0  2.0   2.0  None  None  None  None  24155

2019:10:05-00:00:00  222.63  224.51  217.35  220.59  224.63  226.51  219.35  222.59    2.0  2.0  2.0   2.0  None  None  None  None  15863

2019:10:06-00:00:00  220.58  222.86  217.54  219.96  222.58  224.86  219.54  221.96    2.0  2.0  2.0   2.0  None  None  None  None  19415

2019:10:06-23:00:00  219.94  229.79  214.72  228.05  221.94  231.79  216.72  230.05    2.0  2.0  2.0   2.0  None  None  None  None  33125

2019:10:07-23:00:00  228.06  238.31  228.05  231.62  230.06  240.31  230.05  233.62    2.0  2.0  2.0   2.0  None  None  None  None  32501, 'instrumentType': 'CURRENCIES', 'allowance': {'remainingAllowance': 9992, 'totalAllowance': 10000, 'allowanceExpiry': 602836}}

 

And here's what it looks like around April-5 2020

Quote

 

Response from IG REST API

{'prices':                         bid                             ask                         spread                  last                          

                       Open    High     Low   Close    Open    High     Low   Close   Open High  Low Close  Open  High   Low Close  Volume

DateTime                                                                                                                                  

2020:04:02-23:00:00  225.78  248.33  224.26  236.75  227.78  250.33  226.26  238.75    2.0  2.0  2.0   2.0  None  None  None  None  106597

2020:04:03-23:00:00  236.81  238.76  229.59  234.14  238.81  240.76  231.59  236.14    2.0  2.0  2.0   2.0  None  None  None  None   47901

2020:04:04-23:00:00  234.15  241.81  233.12  235.39  236.15  243.81  235.12  237.39    2.0  2.0  2.0   2.0  None  None  None  None   59462

2020:04:06-00:00:00  235.37  245.53  224.56  243.58  237.37  247.53  226.56  245.58    2.0  2.0  2.0   2.0  None  None  None  None   65707

2020:04:07-00:00:00  243.57  264.96  241.44  258.13  245.57  266.96  243.44  260.13    2.0  2.0  2.0   2.0  None  None  None  None   84988

2020:04:08-00:00:00  258.14  279.18  245.70  259.45  260.14  281.18  247.70  261.45    2.0  2.0  2.0   2.0  None  None  None  None  103281

2020:04:09-00:00:00  259.48  273.91  252.52  261.94  261.48  275.91  254.52  263.94    2.0  2.0  2.0   2.0  None  None  None  None   95054

2020:04:10-00:00:00  261.95  262.39  231.19  235.09  263.95  264.39  233.19  237.09    2.0  2.0  2.0   2.0  None  None  None  None   97915, 'instrumentType': 'CURRENCIES', 'allowance': {'remainingAllowance': 5197, 'totalAllowance': 10000, 'allowanceExpiry': 253266}}

 

So two questions:

1. Why does this happen? UCT is UCT - 00:00:00 corresponds to the start of the day for a 24 hr period. I don't know why it would shift just because we're in DST in AEST.  Is it that the actual reference point for a 24 hr period is 10am -10am AEST, and when DST starts, the UTC adjusts? And then the actual daily/24hr data then 23:00:00 UCT to 23:00:00 UCT? 

2. So you end up with dates within DST shifted by 1 day, which can be fixed post-REST-API - but is there another way to deal with this within the API itself?

Thanks for any advice...

Steven.

Link to comment

5 answers to this question

Recommended Posts

  • 0

Hi Steven, I just found this because I was looking up about daylight saving. 

The English have British Summer Time BST which is UTC+1, so the market adjusts. It also reverts back to UTC+0 when it goes back in time. 

Daylight saving also occurs in NY, so they too move forward in Summer by 1 hour and back in Winter. 

Personally it does my head in, especially when my timezone doesn't change. 

Link to comment
  • 0
30 minutes ago, majicktrader said:

Hi Steven, I just found this because I was looking up about daylight saving. 

The English have British Summer Time BST which is UTC+1, so the market adjusts. It also reverts back to UTC+0 when it goes back in time. 

Daylight saving also occurs in NY, so they too move forward in Summer by 1 hour and back in Winter. 

Personally it does my head in, especially when my timezone doesn't change. 

Every programming language has a core method to convert datetimes to local times.

Python: https://stackoverflow.com/questions/4770297/convert-utc-datetime-string-to-local-datetime

c#: https://docs.microsoft.com/en-us/dotnet/api/system.timezoneinfo.converttimefromutc?view=netcore-3.1

You don't have to deal with time zones yourself. 

  • Sad 1
Link to comment
  • 0
On 14/10/2020 at 17:50, majicktrader said:

The English have British Summer Time BST which is UTC+1, so the market adjusts. It also reverts back to UTC+0 when it goes back in time. 

Daylight saving also occurs in NY, so they too move forward in Summer by 1 hour and back in Winter. 

Personally it does my head in, especially when my timezone doesn't change. 

Thanks.

But why does IG do this? UTC is universal, its a fixed notion of time. UTC itself shouldn't/doesn't change because of daylight savings etc.

Is it the case that IG 24 windows are timed according to a fixed local time in London? Or Sydney? And therefore the UCT changes?

Still confused :)

Link to comment
  • 0
On 14/10/2020 at 18:24, jlz said:

Every programming language has a core method to convert datetimes to local times.

Python: https://stackoverflow.com/questions/4770297/convert-utc-datetime-string-to-local-datetime

c#: https://docs.microsoft.com/en-us/dotnet/api/system.timezoneinfo.converttimefromutc?view=netcore-3.1

You don't have to deal with time zones yourself. 

Thanks Jlz. I've used the arrow lib, it's excellent.

My assumption is that the timestamps coming out of IG are UCT. So I translate the UCT to my local time zone (Melbourne, Aus). 

My confusion is more around my previous response, why does the UCT time shift change, if it's UTC. It must be because ultimately that UCT is based on some other timezone. 

Edited by smx01
Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • General Statistics

    • Total Topics
      20,104
    • Total Posts
      88,179
    • Total Members
      69,098
    • Most Online
      7,522
      10/06/21 10:53

    Newest Member
    Protato
    Joined 04/10/22 18:59
  • Posts

    • Stocks: AAPL, AMZN, NVDA, TSLA, GOOGL, BRK.B, SQ, META, NFLX, ENPH, MSFT, BAC, JPM. Elliott Wave  SP500 US Stock Bear Market: AMZN, AAPL, NVDA, TSLA, GOOGL, BRK.B, SQ, META, NFLX, ENPH, MSFT, BAC, JPM, GS. Elliott Wave Technical Analysis  Stock Market News: Stocks rise, however the move up continue to move up on lower volume, divergence and corrective bear market rally, that should complete in the next trading session Stock Market Summary Elliott Wave Count: Elliott Wave Analysis - (iv) of c) of 4 of (1) Analysis US Stocks: Tesla TSLA, Amazon AMZN, Nvidia (NVDA), Apple AAPL, Microsoft MSFT, Berkshire Hathaway (BRK/B),Block, Inc (SQ), Meta Platforms, Netflix (NFLX), Enphase (ENPH), Alphabet GOOGL. XFL Finance Sector ETF, JPMorgan JPM & Bank of America BAC, Goldman Sachs Group Inc (GS) Stock Market Trading strategies: Looking to short the top and turn in the trading session Video Chapters 00:00 SP500 06:58 Apple (AAPL) 12:22 NVIDIA (NVDA) 14:40 Amazon (AMZN) 18:35 Meta Platforms (META) 20:17 Netflix (NFLX)  21:41 Enphase (ENPH) 24:14 Tesla (TSLA) 27:59 Alphabet (GOOGL)  30:35 Microsoft (MSFT) 31:31 Berkshire Hathaway (BRK.B) 32:33 Block Inc. (SQ)  33:38 Banks JPM, GS 36:14 End. Thanks for supporting! US Stocks: Basic Elliott Wave Counts Apple AAPL  Elliott Wave 4  Corrective rally Amazon AMZN  Elliott Wave 4 Nvidia (NVDA) Elliott Wave  ii) Berkshire Hathaway BRK/B Elliott Wave 4 Alphabet GOOGL  Elliott Wave 4 Meta Platforms FB Elliott Wave iv) Netflix (NFLX)  Microsoft MSFT  Elliott Wave 4 Tesla TSLA  Elliott Wave iii) Square SQ /Block Inc. Elliott Wave 4 JPMorgan Chase (JPM) Elliott Wave 4 Analyst Peter Mathers TradingLounge™ Australian Financial Services Licence - AFSL 317817 Source: tradinglounge com  
    • US DOLLAR TALKING POINTS: The US Dollar has dropped by as much as 3.3% from the high that was set last Wednesday and many are asking if the USD has topped. Given how aggressively overbought the greenback had become, this retracement looks to be a correction in the trend with no evidence yet of anything larger. However – price is approaching some major supports and performance around those levels will be key for determining near-term strategy. Of particular importance is the fact that the US Dollar is a composite of global currencies so the bigger question is when sell-offs in EUR/USD or GBP/USD might return and, in a related item, how will USD/JPY hold up with the pair pegged to the 145 level thought to be the line-in-the-sand for Japan’s Ministry of Finance? Oct 4, 2022 | Full article on DailyFX James Stanley, Senior Strategist
    • EUR/USD and GBP/USD edge up while USD/JPY falters at ¥145.00 again A pause in the dollar rally sees USD/JPY stuck below recent highs, but both the euro and sterling have made gains in early trading against the greenback.    Chris Beauchamp | Chief Market Analyst, London | Publication date: Tuesday 04 October 2022  EUR/USD rebound continues After falling to a twenty-year low last month, the euro continues to rebound against the US dollar with EUR/USD. This still looks very much like a counter-trend bounce that sees the price head back to the 50-day simple moving average (SMA), currently $1.1018 and then begins to falter. This would be in line with previous bounces since April, all of which have found it impossible to hold above the 50-day SMA. This move could see the price head back to $1.01. Further targets lie at $1.02 and then $1.0374. Source: ProRealTime GBP/USD recovers $1.13 The GBP/USD continues to defy the doomsters with a recovery above $1.13. Its own counter-trend bounce remains intact, and if previous bounces are any guide there is still some potential for upside, even if it only reaches the 50-day SMA. Like EUR/USD, the pound is still making lower highs and lower lows against the dollar, with the previous peak at $1.176 marking out the initial target for this bounce. Both stochastics and moving average convergence/divergence (MACD) have room to move up to support this move, but selling the rallies still appears to be the approach here for this downtrending market, although it looks like for now there is still a desire to push the pair higher in the short-term. Source: ProRealTime USD/JPY stalls below ¥145.00 It does look like ¥145.00 is the ceiling in USD/JPY for the time being – repeated attempts to break higher have come to naught, despite the continued strength of the US dollar. Indeed, it is perhaps precisely that we need a pullback in order for the trend to revive. The pair rallied hard from the August-lows, and now sits at some distance from the 50-day SMA (currently ¥139.15). A pullback towards this level might ‘clear the air’, create a higher low and provide a springboard for fresh bullish momentum. Source: ProRealTime
×
×
  • Create New...