Apple Acquires Weather App Dark Sky

[Analog Kid]

One Raspberry Pi uses an Arduino with a custom shield (a WxShield, long since discontinued) to collect temperature/pressure/humidity data from a handful of commercially available sensors inside/outside the house and send the data (via MQTT) to another Pi that maintains a database of weather data. This second Pi also pulls down local data from Dark Sky (via their API) every five minutes, and publishes both the in-house and Dark Sky data (and some historical summaries) to the local network (using MQTT). Two other Pi's read that data and display it continuously on attached touchscreens - these "front-end" Pi's display NTP-sync'd clocks and weather information, and also serve as touchscreen interfaces for my Hue lights (yet another Pi is merely a very accurate clock). It’s all color-coded to heck, so I can quickly get a sense of the conditions from across the room, even if I can’t completely read the display (e.g. the temperatures change color in 5° bands, from purple around freezing, across the spectrum, up to light red above 100°). And I put a light sensor on one of the Pi's, which is sampled every 15 seconds and used to adjust the display brightness on the 3 Pi's that have screens. I don't have large outside sensors like an anemometer attached, so, for instance, the wind information comes from Dark Sky (along with the forecasts). Dark Sky, in turn, is getting it from a traditional PWS (personal weather station) that's about a mile away from my house - good enough for my purposes. The system also periodically uploads screenshots and data to a website of mine, so I can keep track of it remotely. It's been an off-and-on side project since 2016, though the vast majority of development was 2016-2018, because I already added most of the things I wanted. I still tinker with it from time to time. And, since all the data is sent back and forth in standard formats, I could replace various bits and have the rest continue to function (if, say, I put an anemometer up outside). And this is the kind of thing I'll have to do before Dark Sky's API goes away - survey the other providers, select one (possibly weather.gov), and build code to collect data from them and massage it into the format I need (and, along the way, find numerous assumptions I've made while getting too cozy with Dark Sky's data). It'll mean changing a few thousand lines of code.

Here's a screenshot of the main touch panel display (the stripe down the right edge are touch controls for the Hue lights):

View attachment 902863

And this is a web page that (aside from the sunrise/set info at the bottom) is essentially a compact visual translation of the interesting bits of the Dark Sky forecast information:

View attachment 902868

All told it's about 15k lines of Python 3, and involved a lot of fun learning and exploration and going down all sorts of interesting rabbit holes (e.g. the WxShield board came with a huge C# GUI program to read and display the data - they'd intended it as a Windows-only thing - I translated about 4k lines of C# into Python and simmered it down to just the parts I needed for data extraction from the Arduino in a "headless" environment - now, instead it collects the data, translates it into usable form and transmits it via MQTT as it arrives - every 20 seconds or so for the last 4 years, aside from a couple of power failures).

Respect.

 

[boston04and07]

Awful news, I loved using Dark Sky on my desktop for years and now Apple is killing it. Hopefully they relaunch a desktop version with Apple branding but I won’t be holding my breath.

I kind of think they will. There’s already a Dark Sky iPad app and I can’t imagine Apple putting all this time and energy into relaunching the Weather app without an iPad version. Once they do that, it makes sense that it’ll come to the Mac via Catalyst, like the Stocks and Voice Memos apps did when they were redone.

 

[LincolnsiPod]

I'm curious to what people will do for weather API alternatives. With Wunderground API shut down a while back and soon Dark Sky, what's left?

 

[CarlJ]

I'm curious to what people will do for weather API alternatives. With Wunderground API shut down a while back and soon Dark Sky, what's left?

There's some interesting discussion in the comments at Hacker News: Apple acquires Dark Sky.

I'm considering going directly to weather.gov. Sounds like they have a restful JSON interface that should be workable, with some effort. Also sounds like their web interface is purposefully antiquated due literally to lobbying from Accuweather (and possibly others), who would rather you pay to use their services to access the government-funded data, instead of using the the government-funded data directly. This is akin to tax prep companies that lobbying to keep the tax laws complicated to keep themselves in business. Not a fan of intentional waste and middlemen that don't perform some genuinely useful purpose.

 

[AZhappyjack]

I'm curious to what people will do for weather API alternatives. With Wunderground API shut down a while back and soon Dark Sky, what's left?

I know nothing of Foreca, ClimaCell, Aeris Weather or MeteoGroup, but these are the options within Carrot Weather.

 

[victorm1]

One Raspberry Pi uses an Arduino with a custom shield (a WxShield, long since discontinued) to collect temperature/pressure/humidity data from a handful of commercially available sensors inside/outside the house and send the data (via MQTT) to another Pi that maintains a database of weather data. This second Pi also pulls down local data from Dark Sky (via their API) every five minutes, and publishes both the in-house and Dark Sky data (and some historical summaries) to the local network (using MQTT). Two other Pi's read that data and display it continuously on attached touchscreens - these "front-end" Pi's display NTP-sync'd clocks and weather information, and also serve as touchscreen interfaces for my Hue lights (yet another Pi is merely a very accurate clock). It’s all color-coded to heck, so I can quickly get a sense of the conditions from across the room, even if I can’t completely read the display (e.g. the temperatures change color in 5° bands, from purple around freezing, across the spectrum, up to light red above 100°). And I put a light sensor on one of the Pi's, which is sampled every 15 seconds and used to adjust the display brightness on the 3 Pi's that have screens. I don't have large outside sensors like an anemometer attached, so, for instance, the wind information comes from Dark Sky (along with the forecasts). Dark Sky, in turn, is getting it from a traditional PWS (personal weather station) that's about a mile away from my house - good enough for my purposes. The system also periodically uploads screenshots and data to a website of mine, so I can keep track of it remotely. It's been an off-and-on side project since 2016, though the vast majority of development was 2016-2018, because I already added most of the things I wanted. I still tinker with it from time to time. And, since all the data is sent back and forth in standard formats, I could replace various bits and have the rest continue to function (if, say, I put an anemometer up outside). And this is the kind of thing I'll have to do before Dark Sky's API goes away - survey the other providers, select one (possibly weather.gov), and build code to collect data from them and massage it into the format I need (and, along the way, find numerous assumptions I've made while getting too cozy with Dark Sky's data). It'll mean changing a few thousand lines of code.

Here's a screenshot of the main touch panel display (the stripe down the right edge are touch controls for the Hue lights):

View attachment 902863

And this is a web page that (aside from the sunrise/set info at the bottom) is essentially a compact visual translation of the interesting bits of the Dark Sky forecast information:

View attachment 902868

All told it's about 15k lines of Python 3, and involved a lot of fun learning and exploration and going down all sorts of interesting rabbit holes (e.g. the WxShield board came with a huge C# GUI program to read and display the data - they'd intended it as a Windows-only thing - I translated about 4k lines of C# into Python and simmered it down to just the parts I needed for data extraction from the Arduino in a "headless" environment - now, instead it collects the data, translates it into usable form and transmits it via MQTT as it arrives - every 20 seconds or so for the last 4 years, aside from a couple of power failures).

If you don’t mind me asking, what are you using this for? Or is it just a fun hobby project?

 

[CarlJ]

If you don’t mind me asking, what are you using this for? Or is it just a fun hobby project?

Using it for? Why, it's telling me the weather and the time, and occasionally controlling the lights (although that falls to Siri more of the time now). (I live in an old house and it takes some effort to heat and cool - I want to be able to tell at a glance if I should open the windows and turn on the fans. Plus, I like to see information displayed well - I'm naturally curious.)

I'm a software developer. I've been programming computers since the age of 13, not because it seemed like a good career, but because I couldn't not do it - it's endlessly fascinating to me.

This project grew out of dissatisfaction with commercially available weather stations and clocks (I wanted time that was always exactly right, even across DST changes, and I wanted weather I could read from across the room that included multiple temperatures at once - I'd had commercial "weather stations" where you could see one or two temperatures at once, and cycle through others - not good enough). I also wanted to play with UI design for such a device, and wanted color used meaningfully, to impart information, rather than as mere decoration. And I wanted to get fluent in Python. And I wanted to experiment with MQTT (a publish/subscribe protocol for sharing data among IoT devices). And I wanted this same information displayed in multiple places. And I wanted a better way to control my Hue lights (pre HomeKit and Siri, one pretty much had to open the app to change anything). It started out rather simple, and grew in size and scope over several years. It was all done to fill a yearning of mine (several, actually).

I've had friends suggest I should make a product out of the whole thing. I'm not really interested. It wasn't designed to be produceable. One of these days I'll get the code all up on github (I set up a page for it long ago, which explains how it works in more detail, but have yet to put up the code).

 

[victorm1]

Using it for? Why, it's telling me the weather and the time, and occasionally controlling the lights (although that falls to Siri more of the time now). (I live in an old house and it takes some effort to heat and cool - I want to be able to tell at a glance if I should open the windows and turn on the fans. Plus, I like to see information displayed well - I'm naturally curious.)

I'm a software developer. I've been programming computers since the age of 13, not because it seemed like a good career, but because I couldn't not do it - it's endlessly fascinating to me.

This project grew out of dissatisfaction with commercially available weather stations and clocks (I wanted time that was always exactly right, even across DST changes, and I wanted weather I could read from across the room that included multiple temperatures at once - I'd had commercial "weather stations" where you could see one or two temperatures at once, and cycle through others - not good enough). I also wanted to play with UI design for such a device, and wanted color used meaningfully, to impart information, rather than as mere decoration. And I wanted to get fluent in Python. And I wanted to experiment with MQTT (a publish/subscribe protocol for sharing data among IoT devices). And I wanted this same information displayed in multiple places. And I wanted a better way to control my Hue lights (pre HomeKit and Siri, one pretty much had to open the app to change anything). It started out rather simple, and grew in size and scope over several years. It was all done to fill a yearning of mine (several, actually).

I've had friends suggest I should make a product out of the whole thing. I'm not really interested. It wasn't designed to be produceable. One of these days I'll get the code all up on github (I set up a page for it long ago, which explains how it works in more detail, but have yet to put up the code).

Fascinating! It’s very impressive how your mind works. I asked for one main reason and that’s because some of my former colleagues and I had invested significant time and energy into developing machine-learning-based algorithms to programmatically predict the weather to the greatest accuracy possible, as far into the future as possible.

One of the key challenges that we ran into can be illustrated by the simple and common maxim of “garbage in, garbage out.” In other words, the source data on which we based our forecast - namely, (current) wind speed, air temperature, humidity, traffic, human movement, etc. was all fairly rough and wasn’t nearly sufficiently reliable to be usable for our purposes. To reiterate, this is CURRENT weather condition data not projected data.

In addition to this, the reason why we set out developing our own forecasting tool was because we found both the consumer- and enterprise-grade weather modelling solutions utterly lacking both in terms of inputs and (naturally enough) in terms of the data that their models output. If memory serves, their rough hit rate on even the most general factors (+- 10°, rain/sun, wind speed within 10~ knots) was about 50%.

The solution is still in existence, as far as I know but I was absolutely floored by just how bad the status quo is in terms of weather predictions.

No word on how accurate the model we worked on is today, however…

 

[CarlJ]

I'm curious to what people will do for weather API alternatives. With Wunderground API shut down a while back and soon Dark Sky, what's left?

More possible answers (I'm coming at this from the angle of having long used Dark Sky's API as part of the weather station in my living room - it's not a product, it's a one-off - where I only need a handful of calls an hour in total - some of my perspective won't scale to other developers):

• There's an article from last year called "5 Best Free and Paid Weather APIs", that has a few possibilities. None of them seem to match Dark Sky's granularity of results (every minute for 60 minutes, every hour for 48 hours, every day for 7 days). But, with that limitation, there are several free options (OpenWeatherMap and Yahoo Weather are possible). Others start at $25/mo (I really don't want to pay $300/yr - they're talking in terms of 60 API calls a minute, I just want a dozen calls an hour).
• I'm going to take a serious look at using weather.gov's API (and/or here). Free, but doesn't have all the post-processing voodoo that someone like Dark Sky adds to get the more granular data.
• I've long considered having a Mac Mini hooked up to my TV and running 24/7 (mostly headless) as a server. If one of the results of the Dark Sky purchase is that Apple builds a weather API into macOS, then I will seriously consider getting that Mac Mini, and then write some Swift code that queries it every 5 (or 15) minutes and publishes the results on my local network, in JSON format, using MQTT, so I can use the data just as I do now. Advantage here is the granularity of the resulting data may be a close match for what Dark Sky is giving out now. This would definitely not scale for anyone else's uses in the way I'd consider using it (grabbing and republishing), but it's possible that Apple will make an API like this available on macOS/iOS - in that case, an iOS-only weather app could use it (but it would be useless to a iOS/Android cross platform app).

[automerge]1585807085[/automerge]

Fascinating! It’s very impressive how your mind works. I asked for one main reason and that’s because some of my former colleagues and I had invested significant time and energy into developing machine-learning-based algorithms to programmatically predict the weather to the greatest accuracy possible, as far into the future as possible.

Interesting. Sort of coming at the same problem from the other side. What you worked on would have the potential to become something like Dark Sky's back-end (assuming their version actually works as advertised). Ideally you'd want a whole bunch of data points, like the huge network of Personal Weather Stations that Weather Underground worked up... and then sold off to IBM. But, ideally, you'd want something more rigorous than "random people buy random PWS's and set them up with varying skill and plug them into the network". If you had a thousand PWS's from a short list of known models, scattered across a city where you knew precisely where they were and that they were all set up to the same specification (siting, not under a tree, not in the shade, not next to a tall building, etc.), you could get some really solid data to base your models on.

The furthest I've ever gone on modeling is... I have a historical database going back several years, with Dark Sky's numbers for predicted temperatures for my location (based off their model and several nearby PWS's)... each hour, I store their predicted numbers for 0..23 hours in the future, and each 15 minutes, I store an average of all the local readings from my outdoor sensor over the past 15 minutes. Each morning, I generate a set of offsets, showing, for each hour of the day, the difference between their predictions (for 0..23 hours in the future), and the values I actually recorded during the resulting hour, and then I break these down in... if I recall correctly, 5 degree bands.

This gives me a multidimensional array I can do lookups in, of the form, "IF, <18> hours in the future, Dark Sky says that during the <10am> hour it'll be some temperature between <60 and 65> degrees, THEN I need to add <2.1> degrees to that to reach the likely actual temperature at my location." (I've got a script to generate a set of heatmaps of a sort, showing the necessary temperature compensation, per band, per hour, for each hour into the future.) Anyway, I use this array to filter their most recent predictions for the coming 23 hours (adding or subtracting the appropriate amount), and then feed this into the code I use that ultimately predicts the highest and lowest temperatures likely to occur in the coming 24 hours. On the screenshot I posted, this appears as "70° @ 1p, 56° @ 5a". (Yet one more example of a whole ton of code invested in making a slight tweak to something on the display, rather than just going with the far simpler default choice - the system is littered with these - that's why it's so much code.)

 

[bobob]

I learned some new information from the following post on the discussion thread on Reddit on this topic:

"It's a privacy move because Dark Sky is one of the only weather services that doesn't sell your location data. The current stock weather app pulls data from IBM/The Weather Channel which sells your location data.​

​

Also because the Dark Sky API pulls data from primary weather sources, so Apple will have full primary weather data without having to rely on 3rd parties that sell location data.​

​

Source: I researched literally every weather app on iOS that was any good before I settled on Dark Sky for their privacy policy (and awesome service)."​

 

[Abazigal]

I kind of think they will. There’s already a Dark Sky iPad app and I can’t imagine Apple putting all this time and energy into relaunching the Weather app without an iPad version. Once they do that, it makes sense that it’ll come to the Mac via Catalyst, like the Stocks and Voice Memos apps did when they were redone.

Weather is something that seems to make more sense on the Apple Watch compared to other form factors. Think about it - I want timely weather updates that are contextually useful to me (ie: my watch buzzing me with it’s about to rain outside, so I can make an early move or get an umbrella along the way), so it stands to reason that the best weather app is the one which I don’t have to open.

An ipad or even Mac doesn’t make sense, given the way I envision it being used. It won’t stop Apple from making one for every device just because they can, but I just don’t see a tablet weather app being used all that much.

 

[Justanotherfanboy]

Weather Forecast and Weather Forecast Accuracy for Your City

5-day weather forecasts for the United States in a nice, easy to read format, AND the 1- to 3-day out accuracy of other weather forecasters (Accuweather, The Weather Channel, MyForecast.com, Intellicast, and the National Weather Service) including links to their forecasts.

www.forecastadvisor.com

You can go to this site and enter your zip code to compare the accuracy of the various weather providers. For my zip in California Dark Sky does not do so well. Might be better where you live.

View attachment 902561

Huh?
That post/link breaks my brain!

If I understand correctly, the strength in Dark Sky is microlocation... meaning whereas the weatherman can only feed me data from their equipment some 10-15 miles away- Dark Sky can include information captured from the Raspberry Pi powered home brew weather station my next door neighbor runs.
Obviously it’s going to be more accurate!!!

How in the hell could you then look only at that data from 15 miles away & tell me how accurate my predictions were at my house??
That makes zero sense. If the providers of that data you linked had accurate data for my house... I’d be using them! Since they have ZERO data from my house, trying to estimate how accurate Dark Sky is at my house seems like an exercise in futility.

I use both Dark Sky & Wunderground... & may be confusing a few features between the two; but I believe both use your onboard barometer to more accurately calculate rain. The “light drizzle starting in 8 minutes” type of notifications are crazy accurate for me & certainly cannot be replicated by equipment miles from my location.

 

[ipponrg]

Weather is something that seems to make more sense on the Apple Watch compared to other form factors. Think about it - I want timely weather updates that are contextually useful to me (ie: my watch buzzing me with it’s about to rain outside, so I can make an early move or get an umbrella along the way), so it stands to reason that the best weather app is the one which I don’t have to open.

An ipad or even Mac doesn’t make sense, given the way I envision it being used. It won’t stop Apple from making one for every device just because they can, but I just don’t see a tablet weather app being used all that much.

People can have the tablet weather app in the background. It’d be used automatically.

 

[anthogag]

I like Apple's Weather app. It would be better if the icon is animated (E.g. shows rain if raining) and shows current temp right there in the icon. Weather info should also be on the lock screen.

 

[dogslobber]

Still 4 bucks in the app store. When will it be FREE?

 

[Vjosullivan]

I kind of think they will. There’s already a Dark Sky iPad app and I can’t imagine Apple putting all this time and energy into relaunching the Weather app without an iPad version.

It wouldn't surprise me in the least that their weather app would stay unchanged. I could quite easily imagine them changing the data source of their existing weather apps to Dark Sky and leaving it at that.

 

[AustinIllini]

I absolutely hate this. Apple is going to strip this to nothing which is going to kill Dark Sky as we know it

 

[boston04and07]

An ipad or even Mac doesn’t make sense, given the way I envision it being used. It won’t stop Apple from making one for every device just because they can, but I just don’t see a tablet weather app being used all that much.

I couldn’t disagree more. My iPad is my most used device. I check the weather on it all the time. A nice stock weather app that’s accurate would be used all the time (by me).

 

[CarlJ]

I couldn’t disagree more. My iPad is my most used device. I check the weather on it all the time. A nice stock weather app that’s accurate would be used all the time (by me).

It may also be the case that they’ve limited it to iPhones thus far because it actually costs them money (paid to Yahoo or whoever) for the service, and they use it as a (minor) selling point on the iPhone. With Dark Sky in-house, that per-use cost will go away for them, so they may be more inclined to offer it on all devices.

 

[Abazigal]

I couldn’t disagree more. My iPad is my most used device. I check the weather on it all the time. A nice stock weather app that’s accurate would be used all the time (by me).

The point is that I shouldn’t have to. For example, I reply to this post via Tapatalk. It makes more sense for the app to notify me when someone replies to this post, rather than me having to keep refreshing this page to see if anyone has responded.

And the Apple Watch is the one device you will have on you all day (possible even moreso than a phone, and definitely more than a tablet). Context-sensitive information like this make more sense on a wearable device.

That said, if it’s data Apple needs to power said app, then it definitely makes sense to make it available on as many devices as possible.

 

[OUsooner08]

Depends on where you live ... if your weather is 'boring' then it's very accurate, but if your the weather where you live changes by the hour then of course it will be less accurate.

I live in a state with rapidly changing weather and Dark Sky has worked just fine for me.

 

[Yuck9]

I hope this doesn’t mean that Apple will get rid of the design of the stock weather app. I have yet to find another app that provides the weather data I need but in a simple, clean, non-complicated design.

All the other weather apps are just a hot garbage mess of complications and ads (depending on what app) that make things too cluttered and complicated.

They will you can count on that. Does FUBAR ring a bell ? I've removed it and took a few dollar loss. Rather remember it the way it was and not the way it will be.

 

[dilbert99]

Our goal has always been to provide the world with the best weather information possible, to help as many people as we can stay dry and safe

Selling to Apple and stopping sale of this app on Android hardly achieves that goal...

 

[Analog Kid]

Plus, I like to see information displayed well - I'm naturally curious.

Obviously nothing to rival your custom setup for your own home, but I’ve been running Seasonality for a few years and love it’s data presentation. Different purpose though— it’s not meant for at-a-glance consumption.

Seasonality Core

All of the weather. None of the nonsense.

getseasonality.com

The iPhone interface is a bit constrained. iPad is a reasonable compromise. It really shines for me on Mac.

Not a lot of development in the past year or so, but it’s also stable and doesn’t seem to need much. No subscription, at least on Mac (can’t remember for iOS).

 

[CarlJ]

Obviously nothing to rival your custom setup for your own home, but I’ve been running Seasonality for a few years and love it’s data presentation. Different purpose though— it’s not meant for at-a-glance consumption.

The iPhone interface is a bit constrained. iPad is a reasonable compromise. It really shines for me on Mac.

I like Seasonality. I have it on my iPad, iPhone, and Apple TV. Didn't know there was a Mac version. I remember anxiously awaiting the iPhone version and then being somewhat disappointed with the result. These days, for forecasts, I mostly just open up the web page that my app updates.

We’ve been having a bit of rain lately, which spurred me to add a bit of code to the forecast web page. Now, in the per-hour display, at the top, in addition to a circle with a line curving around showing the likelihood of rain (complete circle = 100%), the circle itself “fills up”, to show the predicted amount of rain (full circle = 1/4”). Kinda neat to see the predictions of a storm growing more intense and then tapering off.