Skip to content

Changelog

For a RSS feed of these changes, subscribe using this link: https://github.com/alexander0042/pirateweather/commits/main.atom.

Version 2.0
  • May 30, 2024, API Version 2.0.9
    • Adjust the icon calculation approach to fix issues with a small number of very rainy hours per issue #236.
    • Changed the icon/ summary to use 4 am to 4 am for calculations to follow the Dark Sky approach per discussion #239.
  • May 21, 2024, API Version 2.0.8
    • Yet more error handling code to handle cases when model input files are misformed.
  • May 15, 2024, API Version 2.0.7
    • New error handling code to handle cases when model input files are misformed.
  • May 7, 2024, API Version 2.0.6
    • Fixed a bug in the time specification processing #211.
  • May 7, 2024, API Version 2.0.5
    • Change near surface smoke units to be ug/m3.
  • May 6, 2024, API Version 2.0.4
    • Fixed a bug in the grid index processing #216
    • Enabled CORS support #215
  • April 30, 2024 and May 1, 2024, API Version 2.0.3
    • Fixed a bug in the datetime rounding code failed because of the month rollover as reported in #208
    • Fixed a bug where precipType would sometimes not adjust for the temperature
    • Changed precipProbability to show the chance of precipitation for the rest of the day rather than the day as a whole.
    • These issues were reported in #205
  • April 30, 2024, API Version 2.0.2
    • Fixed a bug where excluding data blocks broke as part of the 2.0 update as reported in #108
  • April 24, 2024, API Version 2.0.1
    • Fixed a bug where cloud cover could go below zero
    • Fixed a bug in the sourceIDX section where negative longitude would display incorrectly.
    • Fixed a bug where precipAccumulation would sometimes show more than four decimal points
Pre-Release Version 2.0
  • Version 2.0l
    • April 18, 2024:
    • Added fire index to currently
    • Added fireIndexMaxTime to daily
    • Fixed a bug where the source time for the HRRR model 0-18 model was incorrect
    • Fixed a bug where the fire index wasn't ingesting
    • Added model lat lon values
  • Version 2.0k
    • April 11, 2024:
    • Added fireIndex, fireIndexMax and fireIndexMaxTime which is behind the version=2 parameter. Data is available in the US and parts of Canada. This was suggested in issue #119
    • Fixed the precipitation probability rounding
    • Added another icon check for no precipitation
    • Fixed negative precipitation intensity
    • Set the accumulation threshold to 0.02 mm for minutely/ hourly
  • Version 2.0j
    • April 9, 2024:
    • Changed the threshold to show the precipitaion icon in the currently block to 0.02 mm/h
  • Version 2.0i
    • April 5, 2024:
    • Fixed an issue where daylight savings time would offset the daily time parameter. This was reported in issue #134
    • Fixed an issue where the hourly time parameter would be offet for fractional TimeZones as reported in issue #32
    • Added dawnTime and duskTime which is behind the version=2 parameter. This was suggested in discussion #144 and issue #154 which was created from the discussion
  • Version 2.0h
    • April 5, 2024:
    • Fixed the remaining bugs from issues #155 and #180
  • Version 2.0g
    • March 19, 2024:
    • Fixed an issue where the currently cloud cover would interpolate from the next hour data to the current hour data.
  • Version 2.0f
    • March 19, 2024:
    • Fixed an issue where the currently cloud cover would interpolate from the next hour data to the current hour data.
  • Version 2.0e
    • March 18, 2024:
    • Fixed an issue where the currently icon and summary would return None instead of an icon
    • Fixed an issue where the currently icon and summary would return clear instead of the correct icon
    • Changed the logic for the currently and minutely preciptiationIntensity to return data even if precipitationProbaility is zero
  • Version 2.0d
    • March 15, 2024:
    • Fixed the currently icon thresholds that occured after changing current cloud cover to a percentage
  • Version 2.0c
    • March 13, 2024:
    • Fixed the daily min/max timing issues (turns out there were several different things that weren't working)
    • Fixed most of the #155 items
    • HRRR subhourly and HRRR 0-18 will always be 1-2 hours off now. I'm trying to ingest the subhourly data as fast as possible, so it gets read as soon as four timesteps are published. The hourly data waits until the 18 hour timestep is out, and then has to process the previous 36 hours of data to create a cohesive timeseries.
  • Version 2.0b
    • March 11, 2024:
    • Excited to announce that the long awaited version 2.0 of the Pirate Weather API is ready for beta testing on the dev.pirateweather.net endpoint!
    • At a high level, there are four main improvements that will impact every request:
      1. Includes a fancy new model from NOAA called the National Blend of Models
      2. Does a way better job of calculating the daily high/ low/ accumulations for the current day
      3. Returns somewhere between 10 and 50 times faster than v1 (this was my favourite to work out). I can't say for sure that it's the fastest weather API out there, but it's definitely in contention now.
      4. Faster data ingest (~5 minutes).
      5. Improved US alert processing.
      6. Nearest storm distance and bearing!
    • As well as several new optional improvements behind a new version=2 querystring parameter, to avoid breaking Dark Sky compatibility:
      1. Short term (~36 hour) smoke forecasts.
      2. Liquid, snow, and ice precipitation types.
      3. Model specific exclusions (exclude=hrrr or exclude=nbm), to facilitate performance comparisons between models.
      4. Returned grid indexes of model results (this seemed small, but since HRRR is in Lambert, it was fairly complex).
  • March 8, 2024
    • Currently block added which uses a mix of NBM/HRRR/GFS/GEFS data
    • Daily block added
    • Alerts block added
    • Icon and summary data points added to all blocks
    • Apparent Temperature now uses data from the GFS model instead of calculating it. This was reported in issue #76
    • Fix issues documented in issue #155
  • February 18, 2024
    • Hourly and minutely blocks have been added
  • February 6, 2024
    • National Blend of Models data is now being shown in the API.
    • Querying by lat/long is now working
    • Nearest storm data and bearings now return data instead of always returning zero. This was reported in issues #6, #91 and #121
    • Development URL is now located at http://piratev2lb-a90c79daaddc2625.elb.us-east-1.amazonaws.com:8000/forecastv2/<APIKEY>/<LAT>,<LON>
  • January 4, 2024
    • Right now, there's not much to look at, since it's only a testing endpoint to make sure that things were flowing; however, it represents a ton of key improvements:
      1. ~100x faster responses
      2. NBM data (although it's not shown yet, it's in there!)
      3. Storm distances and directions
      4. Short term historic data, fixing the daily high/ low issues.
      5. Less interesting, but totally new ingest pipeline that's way more flexible/ maintainable.
    • To access it, check out: https://dev.pirateweather.net/forecastv2/<APIKEY>/<X>,<Y>
    • X and Y are raw grid indexes (I warned it was rough), so keep them between 0-700, and the only data that's returned right now is the -36 hour UTC time for the HRRR model. In the next couple days, I'm going to fix lat/long and expose some of the raw model data (probably current values for everything), so they'll be something to look at!
Version 1.5
  • December 11, 2023: API Version 1.5.6
    • Added error handling for times in the future per issue #122.
    • Improved how short term historic forecasts (<3 days) and processed.
    • Not live yet, but significant progress on V2.0 has occurred, and ingest scripts for NBM + HRRR Smoke are complete, as well as updates to allow for previous observations to be used in processing.
  • August 29, 2023: API Version 1.5.5
    • Dull back-end update: added response compression to reduce a terrifyingly large AWS data transfer bill, and removed old logging statements.
  • July 13, 2023: API Version 1.5.4
    • Fixed a series of rounding issues: #4, #36, and #77.
  • July 6, 2023: API Version 1.5.3
    • Fixed a glitch producing a "snow" precipitation type when very small amounts of precipitation were forecasted per issue #78.
    • Fixed an issue with the icon field returning the night icons in areas with 24 hours of sun per issue #79.
    • Fixed an parsing problem with small negative latitudes per issue #67.
  • May 26, 2023: API Version 1.5
    • Least interesting point update ever, but important nevertheless! Migrated existing keys from using AWS API Gateway to Kong as the back-end layer. This will allow the service to grow past the 10,000 user limit imposed by the AWS Gateway, allows me to add a caching layer to improve performance, provides more AWS availability zone redundancy, and provides some more flexible routing options in the future. As a bonus, Kong returns the number of API requests remaining as a header, addressing issue #54.
Version 1.4
  • March 13, 2023: API Version 1.4.1
    • Fixed a rounding bug introduced in 1.4 which sometimes created issues with the currently block precipitation intensity parameter over the HRRR grid per issue #29.
  • February 28, 2023: API Version 1.4
    • Fixed a long standing bug in the GEFS precipitation intensity for the currently and minutely blocks per issue #24. Outside of the HRRR area, precipitation intensities are now calculated using GEFS, instead of just returning zero!
    • Corrected how midnight is calculated in the Eastern Hemisphere.
Version 1.3
  • January 27, 2023: API Version 1.3.2
    • Fixed an issue created when fixing the HRRR grid point issue that resulted in the HRRR model not being used per this issue.
  • January 26, 2023: API Version 1.3.1
    • Fixed an issue with the time parameter not showing up as an integer per this issue.
    • Fixed an issue where the HRRR model would grab results for the incorrect grid cell when a point very near the edge was requested.
  • January 24, 2023: API Version 1.3
    • Rapid update here- implemented the temperatureHigh and TemperatureLow parameters to match Dark Sky, by using the daily high and nighttime low data.
    • Added the missing windGuestTime.
    • Corrected a bug on all the max/min times for days 2-7.
    • Changed all daily parameters except icon/ summary to be calculated over a 24 hour period from 12:00 am to 12:00 am, instead of 4:00 to 4:00.
    • Fixed a bug with sunrise and sunset times in UTC+x time zones.
Version 1.2
  • January 23, 2023: API Version 1.2
    • Alerts! Finally wrote a processing script to save weather alerts as a NetCDF file, which provides much more detail as well as much faster response times.
Version 1.1
  • December 1, 2022: API Version 1.1.10
    • Quick fix to address a server error when requesting a point on the edge of the HRRR grid.
  • November 25, 2022: API Version 1.1.9
    • Add in a daily average precipitation intensity for compatibility with Dark Sky. Note that this value will always be in either mm or inches, so may not align with accumulation during days with mixed rain and snow.
  • October 28, 2022: API Version 1.1.8
    • Change the interpolation weighting function for GFS/ GEFS model data from inverse distance (1/distance) to inverse distance squared (1/distance^2). This increases the weight of the nearest grid cell, particularly when a forecast point is near the centre of a cell. The goal for this change is to make results more accurate in coastal areas where there can be large differences between land and water cells while keeping the smooth transition from one grid point to the next.
  • September 7, 2022: API Version 1.1.7
    • Update the logic when model data is missing per this issue.
  • March 30, 2022: API version 1.1.6
    • Updated the fog icon to allow it to display at night.
    • Aligned the cloud cover icon with NOAA definitions.
    • Capped the visibility at 10 miles to avoid HRRR/ GFS inconsistency.
    • Corrected a bug causing "-0" to be returned when very small accumulations were forecasted.
    • Corrected the wind icon threshold.
    • Resolves Issue #3.
  • March 25, 2022: API version 1.1.5
    • Changed the visibility threshold for the fog icon to 1 km per OFCM.
    • Part of issue #30.
  • March 22, 2022: API version 1.1.4
    • Fix a bug when requesting data at the edges of the domain per issue #41.
  • March 16, 2022: API version 1.1.3
    • Small performance increase (~0.3 s) by changing the way time zones are calculated to use TimeZoneFinderL. This could result in incorrect time zones sometimes, but since this isn't used as part of the weather details, shouldn't pose a major issue. If highly accurate time zones are required, a new tz=precise url parameter is available.
    • Return elevation data from ETPOP1.
  • February 18, 2022: API version 1.1.2
    • Fix for missing alerts that were missing an issued time.
    • Corrected the accumulation when sleet was forecasted (part of issue #30).
  • February 4, 2022: API version 1.1.1
    • Fix for missing alert description.
  • February 1, 2022: API version 1.1
    • Bugfix and performance release to clean up some things that didn't make it into V1.0.
    • Much (30%) faster API response times by changing how the nearest grid cell lookup is performed. The grid cell is now calculated directly using the HRRR/ GFS grid math, as opposed to reading from an index file. The time zone lookup has also been optimized to only call the lookup function once.
    • Fixed the implementation of the "exclude" option.
    • Fixed the US NWS alerts parser.
    • Per this issue, precipitation intensity is now always returned in mm of water equivalent per hour.
    • Per this issue, the precipitation icon is now trigged by 0.25 mm/h of precipitation, instead of 1 mm/h.
    • Corrected a small bug where a negative precipitation rate could be returned.
Winter 2022
  • Winter 2022
    • Official V1.0 release! These docs have been updated to reflect the current version, but I'll leave the previous version up for reference under the v0.1 header.
    • Changed the data ingest pipeline to use AWS Fargate (thanks sponsors!) improving the resolution by 4x!
    • Added short term historic data via the time parameter.
    • Fixed a long standing issue with wind speeds.
    • Added support for the exclude flag.
    • Moved the documentation over to ReadTheDocs.
    • Published the processing scripts and docker image.
    • Added a version tag to the flags field
  • October 4, 2021
    • Still working on bringing the NBM datasource online, but in the meantime I fixed a couple issues with cloud cover and pressure data responses.
    • The back end of this service is also getting more stable and predictable, so I've raised the free tier to 20,000 API calls/ month.
  • August 17, 2021
    • Fixed how the API returns calls for locations at the edge of the grid, identified here
  • July 26, 2021:
    • Fixed an issue with the uk2 units.
  • June 22, 2021:
    • Major rework of the alerts setup. The old method had more detail about the alerts, but missed any that didn't include coordinate data (which was about half!). These missing alerts were just associated with a NWS region. Luckily, the amazing Iowa State Mesonet has a geojson source for current alerts, and every alert has a polygon attached! The alerts data (still US only) is now pulled from here.
  • June 9, 2021:
    • Added several new variables to the front end website
    • Changed the UV processing factor from 0.25 to 0.4
    • Corrected a sunrise/ sunset timing issue
  • May 25, 2021:
    • Corrected an icon issue, identified here
  • May 20, 2021:
    • Changed the GFS retrieval to interpolate between a weighted average (by 1/distance) of the 9 closest grid cells, instead of just taking the nearest cell. This will help to smooth out some of the sudden jumps in the results.

Time Machine Changelog

  • April 18, 2024
    • Fixed an issue where locations in fractional timezones were not returning an error as reported in #194
  • March 11, 2024
    • Fixed an issue where using local time format returned an Invalid Time Specification error as reported in issue #162
  • January 16, 2024
    • Change the Time Machine to use ERA-5 model data from Google's dataset as AWS removed it. This was reported in #130
      • As a result of the change the Time Machine endpoint only has historical data until May 2023
  • August 29, 2023
    • Fixed an issue where some legacy keys could not consume the Time Machine endpoint reported in #98
  • July 10, 2023
    • Fixed an issue with the timing of a day that rolls over into the next month as reported in issue #86
  • June 19, 2023
    • Fixed an issue where locations in fractional timezones were not returning an error as reported in #74
  • May 27, 2023
    • Fixed an issue where negative longatide and latitude would be parsed as positive. This was reported in #67
  • April 6, 2023
    • Fixed an issue where apparentTemperatureMinTime and apparentTemperatureLowTime were not returning UNIX timestamps. This was reported in issue #46
  • March 20, 2023
    • Fixed an issue where Time Machine data was returning an error for dates after June 30, 2022
    • Fixed the data in the currently block
    • Integers are integered, floats are rounded to 2 decimal points
    • Text descriptions are now much better
    • These issues were reported in issue #18
  • December 23, 2021
    • Fixed an issue where daylight savings time was being ignored as reported in #4
  • August 3, 2021
    • Fixed an issue where a combination of a time zone glitch and an issue with dates that were the last day of a month, since they required two separate calls to the archive.
    • Fixed a couple other issues pertaining to rain/snow units
    • These issues were reported in #3