Google Analytics is a great platform, but it’s user agent detection can be a bit strange. New user agents aren’t back ported, so when support is added for a new browser or OS the stats tend to change dramatically going forward. An example from earlier this year is when I tracked down what Google Analytics meant by the browser “Safari (in-app)”.
More recently I ran through my analytics for May 2013 and saw a Operating System called “Firefox OS“. This is the much-hyped offering from Mozilla and while I am excited to test it out and perhaps develop for it, there are not yet any shipping devices. It’s strange to see traffic for an OS that doesn’t have any real users (especially the amount I saw), so I dug in a bit to see what Google Analytics was doing.
Firefox OS’s user agent
Firefox OS uses a user agent that follows the mold they have been sticking with for years and at launch will be (where X is the version number):
Mozilla/5.0 (Mobile; rv:X.0) Gecko/14.0 Firefox/X.0
The user agent is not in my logs
I grepped through my logs and couldn’t find anything close to a match despite Google suggesting I receive about 2,000 visits a day with Firefox OS. This confirmed for me that Google Analytics and I disagree with what Firefox OS is.
The plot thickens
Using secondary dimensions in Google Analytics I was able to find that the visits were all from the US, the screen resolutions were all 1024×768, Flash was supported (version 11.2 r202), no OS version was available and most interestingly that the network domains all resolved back to yahoo.com. I had previously tracked down some weird issues with Yahoo’s bot showing up in Google Analytics and this was starting to look quite similar.
Google Analytics doesn’t give you any raw data (IP addresses, full user agent, timestamps, etc) so it’s hard to line up with server logs. Using landing pages I was able to find some odd balls and find their corresponding log entries to see if I could find the exact user agent strings. I found the culprit:
Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) NOT Firefox/3.5
Somehow Google Analytics is seeing this user agent as “Firefox OS”. What a pain. You can block it from your reports using Advanced Segments (I detailed this in my post about Inktomi showing up in reports).
The new Yahoo! bot is executing the pages
tl;dr Google Analytics is counting a Yahoo! bot as Firefox OS
Update Aug 1st 2013 – The bot stopped on July 27th 2013, your reports should be fine
Update Aug 15th 2013 – The bot showed up again starting on August 13th 2013, this time with 4-6x the number of visits
Olympus makes its ORF RAW file format viewer software available as a free download, but require that you provide your camera’s serial number before getting the app. I have no idea why this is the case considering the application is only useful if you have .ORF files and those files are only created by Olmpus cameras. There are multiple scenarios where you would want the app and not have the serial number… Say if you sent your RAW files to someone else and they didn’t already have the app, or you were at work and your camera was at home.
The good news is that serial number validation is all client side and the algorithm is quite simple. You can just use any 9 digit number and you can download the viewer software. Even with my camera miles away I was able to download the app.
I run a decently popular website and frequently receive advertising inquiries. I spend money every month to keep the site running and advertising is how I keep it afloat. If it were only that that easy–if you’re not careful it is possible to find yourself swindled by criminals looking to pump malware out to your site’s visitors under the guise of purchasing advertising. And by “purchase” I really mean “agree to purchase”, it’s almost certain that they won’t end up paying you (and there may be fun legal implications if you accept money from people spreading malware on your site).
Major sites have been hacked for this to happen, but it’s easier to just buy some advertising and immediately have your exploit delivered to millions of unsuspecting users. Sometimes it happens to advertising networks which is more frustrating because you don’t have direct control over which ads are displayed or the ability to audit code beforehand. About the only thing you can do is to pick ad networks carefully and keep close tabs on the creative they are delivering. This exact issue has caught some big name sites like Tech Crunch and Cult of Mac recently so don’t think it can’t happen to you.
Google Analytics provides breakdowns by user agent and on Crossword Tracker’s statistics I recently noticed a separate entry for Safari named Safari (in-app). Looking back it actually started exactly on August 21st which is probably when Google updated its user-agent parsing code to separate out this traffic. Safari (in-app) appears to refer to users opening your website in a UIWebView (developer talk for a web browser embedded in an iOS or Mac application). You can customize the user agent in a UIWebView and Google Analytics breaks those out when they can (they do this with Chrome for iOS), but if you leave it to the default it shows up as an iOS device without a defined version of Safari. Previously these visits were counting as “Mozilla Compatible Agent”, but they are now Safari (in-app). Here’s what the chart looks like for iOS traffic just before, during and directly after the change:
I did some digging to find out how Google Analytics would be able to differentiate and here’s what some iOS user agents look like:
Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B143 Safari/8536.25
Home screen app (bookmark saved to homescreen)
Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B143
Chrome for iOS
Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X; en-us) AppleWebKit/536.26 (KHTML, like Gecko) CriOS/23.0.1271.100 Mobile/10B143 Safari/8536.25
Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) 1Password/4.1.2 (like Version/10B143 Mobile/6.1 Safari/8536.25)
window.navigator.standalone property. I unsuccessfully explored the
window.navigator object with the various in-app browsers to see if I could shake out their names. If your app has a UIWebView and you don’t customize its user-agent (like you see Google did with Chrome–CriOS) there doesn’t appear to be a way to categorize the traffic. That’s too bad, I’d like to know what apps people are using.
Apple has hired Advanis to conduct surveys of Apple customers. This seems something of a non-Apple move (Steve Jobs was famous for not asking the customer what they wanted), but does appear to be legitimate. I received one today regarding an iPod that I purchased a couple months back. (As an aside, iPods are definitely not hot right now but I figured I should go ahead and buy an iPod Classic while they’re still around so that I can have my entire music collection in my car. 160GB is a lot bigger than any of the flash options and I could see it being discontinued at any time.)
Interestingly enough I didn’t buy the iPod from Apple directly, I got it through Amazon. Apple must have picked up my details when I synced it with iTunes. It came from MarketResearch@InsideApple.Apple.com and was sent through an Apple server (which is why it is legitimate). Advanis is a Canadian market research firm and other than the URL the survey itself is completely branded by Apple.
Here’s what the email looks like:
For what it’s worth, I did not take the short iPod survey. If Apple’s reading: I bought it because it’s got a ton of storage and I have a feeling you won’t buy able to buy iPods with this much storage for very long.
If you’re a web publisher considering using CPX Interactive to fill advertising inventory I recommend that you do not. CPX has failed to pay me on their own terms and has stopped responding to communication.
CPX was originally quite motivated to get things going, their account reps John Rosa and James Weigel responded nearly immediately as we discussed the deal. The insertion order was sent over quickly and I returned it ASAP. Advertising started and that’s when I stopped hearing anything. We had agreed upon a CPM payment structure, but unmentioned in our discussion was a limitation to US traffic. To be fair they did include this on the IO, but it wasn’t spoken of and it turns out CPX arbitrarily decided to pay 15% of our agreed upon CPM for traffic outside of the US. The account reps that were so eager to get started never mentioned anything about the non-US traffic and the reporting tool they provided has no way to see this traffic that is not providing a return.
At the end of the day a few less bucks isn’t the end of the world, but it was a valuable lesson. However, that’s not why I wouldn’t recommend working with CPX (though it would make me think twice). The real reason I don’t recommend CPX is that I haven’t seen a dime yet despite starting to run their ads three months ago!
CPX wanted to pay on a net-45 basis which is a bit long for my taste but was not too extraordinary (everyone likes to be paid quickly and not have to pay their bills quickly). I agreed to the net-45 terms, but CPX has failed to meet their own payment terms. At the time of this writing CPX has missed their own 45 day mark by a cool 28 days. I pointed this out and since then CPX won’t respond to my emails or answer my calls. Thanks to their long net-45 terms I had run their ads for another month and a half before I knew there would be a problem receiving payment (and before I found out my payments were going to be for much less than I had anticipated because of the international traffic). The real kicker is the ads all seemed to be for digital products sold through ClickBank which is on a two week payment schedule. Net-14 must be nice.
There are more reputable firms to work with, I don’t recommend working with CPX Interactive.
Update: Like magic, the day after this post I received a payment. Supposedly covering August and September, but the payment was sent without a breakdown of how the figures arrived at and they were [significantly] different than both the CPX reporting tool and the original higher figures based straight off CPM. The kicker was getting an email from someone in accounting about how I should have received a payment for $X where $X was yet another figure. So to re-cap: I have a total of what I was expecting from a straight CPM basis, what CPX’s own reporting tool says, what the payment was for and what accounting said the payment was for. This is not encouraging. Supposedly my account rep is “looking into it” and I’ll hear back at a later date. I stand by my earlier statement–there are much more reputable firms to work with than CPX. You can’t wing it, especially when it comes down to money.
Update 2: The small discrepancy above ended up being a timezone issue, despite my using the default timezone and the account rep’s emailed figures matching what I saw in my timezone. Incredibly sloppy on CPX’s behalf.
Update 3: It’s now December 17th and yet again CPX Interactive has missed its payment window (a generous 45 days). I’m still waiting to be paid for October. This is starting to feel like not paying publishers on time is a routine business operation over at CPX Interactive. It will be interesting to see what the payment total is for this time…
Update 4: After following up with all my contacts at CPX, I got paid on December 20th, which would be Net-50. As usual, the amount paid was slightly different than what their reporting system says, but it was just a few pennies which makes it nothing more than a curiosity.
I had a strange problem pop up recently in Mac OS X Mountain Lion where operations to reveal a file in the Finder would not work. I noticed it first in Chrome where I would try and open a downloaded file and could not get to it. Then I tried to reveal a download in Safari and got the same thing (confirming it was not a bug in Chrome). The same issue then appeared in Sublime Text 2, I could not jump to the Finder with my file selected. Same with Adobe Lightroom. iTunes still works, but that’s the only example I found that was not broken. Pretty much everywhere you can “Reveal in Finder” or “Open in Finder” is broken. The craziest part of this thing is it happened both on my laptop and desktop.
It appears I am not alone, which means it’s likely on Apple’s side and is not due to some combination of apps that I use. There is not a permanent fix [yet!], but you can get a temporary reprieve by killing the appleeventsd process:
$ sudo killall -KILL appleeventsd
The underlying bug seems to be unrelated to the action of actually revealing a file in the Finder, but that’s a common operation that demonstrates the issue. According to this report, the problem is applications getting into a state where they cannot send Apple events when the events are addressed by the bunder identifier.
After years of considering it, I finally scooped up my first eInk display powered Kindle. The Amazon Kindle PaperWhite is the latest generation of Kindle and combines the touch screen technology first seen in the Kindle Touch (though it has been improved) with a novel backlight system to facilitate reading in the dark. The light is what suckered me in, I had stayed away from eInk because I frequently read in bed and having to use a book light seemed absurd (not to mention distracting for anyone else). Now with the PaperWhite I can read in full sunlight and in the dark, all without any accessories.
The size is great, if you haven’t used a 7″ tablet before it’s the perfect size for reading. That’s why Apple shrunk the iPad (or slightly enlarging the iPhone I guess, since it came first) to be the same size. You can hold it with one hand and it’s perfect for reading while laying down. After using the Kindle for a bit the [full sized] iPad feels gigantic.
I live in Florida and it’s wonderful to have a device with a screen that looks perfect in direct sunlight. I can’t wait until they figure out how to do the same with non-eInk displays so that phones and regular tablets will be usable outside (or even with polarized sunglasses).
The touch screen on the PaperWhite responds faster than the Kindle Touch and the redraw rate also seems improved. The on screen keyboard is surprisingly usable, though I have only needed to use it a few times for WiFi passwords and a couple searches. I opted for the 3G model so that my reading progress would sync to Whispernet without connecting to WiFi which is often the case if I’m traveling.
The only thing I would change is to put a button on the back to turn the page. It’s slightly annoying to have to touch the front, though usually a thumb can reach and not be too much of a hassle.
I had Google Drive pop up an interesting dialog today asking for my permission to add status indicators to icons in the Finder. I think this is a play to make Drive more like Dropbox, which uses icons to show the status of files in the Finder. I gave my permission and then checked my Google Drive folder to see if anything changed and after a few minutes it looks like I was exactly right. For reference, I’m currently using version 1.5.3654.0684.
They still seem a little buggy because I noticed after dragging that folder of photos into my Google Drive that they all initially had a green “synced” check icon and then all switched to the blue refresh icon and finally each updated back to a check as they were synced. They also don’t appear to update in the background, I had the window out of focus and thought progress was stalled, but clicking on the window updated all the icons to show sync was complete.
I got an iPhone 5 right after getting a Hyundai Genesis and to my dismay the new phone and new car did not work together. I assume other car manufacturers are in the same boat because Apple did not provide any heads up on the new Lightning connector. The Genesis has an auxiliary input as well as a USB input and Hyundai actually included a 30-pin connector cable that oddly fits into both ports. This worked great with any previous iOS device, you get a digital signal right into the head unit and can control playback through the steering wheel and everything else you would expect.
The iPhone 5 does away with the venerable 30-pin connector in favor of the smaller Lightning connector. A Lightning to USB cable is provided in the box for charging and plugging that into my Hyundai Genesis sent the phone into a repeating loop of trying to connect but never making it. Frustrating to say the least, but at least Bluetooth still worked fine so I had music (but no control, over Bluetooth you have to change the songs on the phone and not from the car which is not safe).
I ordered the iPhone 5 along with the $29 Lightning to 30-pin Adapter at the same time, but Apple apparently put more focus on the phone and didn’t start shipping the adapters until last week. It’s now at a 2-3 week shipping time which I assume means they’re selling like hot cakes (no surprise since at this time there are exactly 0 accessories that are compatible with the Lightning connector…) I have finally received the adapter and am happy to report that there appears to be full functionality with my car–the phone shows it has power and playback is controlled through the vehicle. Success. Your mileage may very, all iPod integrations are not the same and it may only be a certain variety that are compatible. All I know is that whatever is in the 2012 Hyundai Genesis is indeed compatible with the new Lightning to 30-pin Adapter Oddly enough, apparently even Hyundai had no idea if it would work or not and were similarly waiting to get their hands on an adapter to find out. Apple really could have made this transition a lot smoother.
Spoiler: the Lightning to 30-pin Adapter is compatibile with the Hyundai Genesis (and likely other models)
Despite the phone now working with my car, I still think I’m just going to go with an iPod Classic and keep it in the center console. 160GB fits my whole music collection and I don’t have to bother with plugging or unplugging anything when using the vehicle. It’s also nice that with the direct connection music starts where I left off and as soon as I start the car (with Bluetooth it defaults back to the radio and if I have to plug in my phone there’s obviously a delay there too).