Help Build a Tool To Track Apple Support Page Changes

How about some code like this? You can put it in your crontab to run daily and email you if anything has changed in the last 7 days.

#!/bin/bash

# Crontab Example:
# ----------------
# This example runs a the job daily at 4:50 am
# 50             4      *       *       *       /Users/bin/apple_support_pages.sh -e                 &> /dev/null

mailit() {
echo "To: $EMAIL_ADDRESS
Subject: $SUBJECT
Content-Type: text/html
<html>
<body>
<pre style=\"font: monospace\">"
local line
while IFS= read -r line; do
   line2=`echo "$line" | sed 's/\r//'`
   printf '%s\n' "$line2"
done
echo "</pre>
</body>
</html>"
}

check_pages() {
  URLs="https://support.apple.com/en-us/HT214116
  https://support.apple.com/en-us/HT214117
  https://support.apple.com/en-us/HT214118
  https://support.apple.com/en-us/HT214119
  https://support.apple.com/en-us/HT214120
  https://support.apple.com/en-us/HT214121
  https://support.apple.com/en-us/HT214122
  https://support.apple.com/en-us/HT214123
  https://support.apple.com/en-us/HT214124"

  CHANGED=0
  TODAY_SECS=`date "+%s"`
  x=0

  # grap the URL and save it to a file
  for URL in `echo "$URLs"`
  do
    let x=x+1
    curl -s -o $TMPFILE1 $URL

    DATE=`grep publish_date $TMPFILE1 | cut -f2 -d'"'`
    DATE2=`date -j -u -f "%m%d%Y" +"%Y-%m-%d" $DATE`
    DATE_SECS=`date -j -u -f "%m%d%Y" +"%s" $DATE`

    let DAYS=(TODAY_SECS-DATE_SECS)/86400

    TITLE=`grep og:title $TMPFILE1 | cut -f4 -d'"'`

    if [ "$DAYS" -le "$CHECK_DAYS" ]
    then
      STAR="  *"
      CHANGED=1
    else
      STAR="   "
    fi
    echo "$x. $STAR $URL - $TITLE - $DATE2"
  done
}

NAME=$(basename "$0")
VER=1
MY_HOSTNAME=`/bin/hostname -s`

d=`date "+%Y-%m-%d - %H:%M:%S"`

# for home brew
if [ -f /opt/homebrew/bin/brew ]
then
  eval "$(/opt/homebrew/bin/brew shellenv)"
else
  eval "$(/usr/local/bin/brew shellenv)"
fi


#################################
# change these to match your configuration
#################################

EMAIL_ADDRESS="YOUR@EMAIL_ADDRESS"
CHECK_DAYS=7
TMPFILE1=$HOME/tmp/apple_support_pages_tmpfile.html
TMPFILE2=$HOME/tmp/apple_support_pages_tmpfile.txt

#################################
#################################


echo "----------------------------------" 												>  $TMPFILE2
echo "$NAME (v. $VER) - $d" 															>> $TMPFILE2
echo "Hostname: $MY_HOSTNAME" 															>> $TMPFILE2
echo 																					>> $TMPFILE2
echo "Apple support pages website check"                                                >> $TMPFILE2

echo 																					>> $TMPFILE2
check_pages                                                                             >> $TMPFILE2
echo 																					>> $TMPFILE2
echo "Note: * means the files has changed in the last $CHECK_DAYS days"                 >> $TMPFILE2
echo 																					>> $TMPFILE2


if [ "$CHANGED" = "1" ]
then
  SUBJECT="Potential Apple support pages changes"
else
  SUBJECT="No Apple support pages changes found in last $CHECK_DAYS days"
fi

cat $TMPFILE2 |  mailit | /usr/sbin/sendmail -t  -F "$MY_HOSTNAME"
cat $TMPFILE2

rm -f $TMPFILE $TMPFILE2
1 Like

Looks interesting! I’ll give it a spin and see what it does. Seems like a great start.

I’ve been fiddling with this. It looks like a bunch dropped yesterday and today.

Date Link
08/12/24 About the security content of iOS 16.7.6 and iPadOS 16.7.6 - Apple Support
08/12/24 About the security content of iOS 17.2 and iPadOS 17.2 - Apple Support
08/12/24 About the security content of macOS Monterey 12.7.4 - Apple Support
08/12/24 About the security content of visionOS 1.1 - Apple Support
08/13/24 About the security content of AirPods Firmware Update 6A326, AirPods Firmware Update 6F8, and Beats Firmware Update 6F8 - Apple Support
08/13/24 About the security content of GarageBand 10.4.11 - Apple Support
08/13/24 About the security content of iOS 16.7.7 and iPadOS 16.7.7 - Apple Support
08/13/24 About the security content of iOS 16.7.8 and iPadOS 16.7.8 - Apple Support
08/13/24 About the security content of iOS 16.7.9 and iPadOS 16.7.9 - Apple Support
08/13/24 About the security content of iOS 17.4 and iPadOS 17.4 - Apple Support
08/13/24 About the security content of iOS 17.4.1 and iPadOS 17.4.1 - Apple Support
08/13/24 About the security content of iOS 17.6 and iPadOS 17.6 - Apple Support
08/13/24 About the security content of iTunes 12.13.2 for Windows - Apple Support
08/13/24 About the security content of macOS Monterey 12.7.2 - Apple Support
08/13/24 About the security content of macOS Monterey 12.7.5 - Apple Support
08/13/24 About the security content of macOS Sonoma 14.2 - Apple Support
08/13/24 About the security content of macOS Sonoma 14.4 - Apple Support
08/13/24 About the security content of macOS Sonoma 14.4.1 - Apple Support
08/13/24 About the security content of macOS Sonoma 14.5 - Apple Support
08/13/24 About the security content of macOS Ventura 13.6.3 - Apple Support
08/13/24 About the security content of macOS Ventura 13.6.6 - Apple Support
08/13/24 About the security content of macOS Ventura 13.6.7 - Apple Support
08/13/24 About the security content of Safari 17.4 - Apple Support
08/13/24 About the security content of Safari 17.4.1 - Apple Support
08/13/24 About the security content of tvOS 17.2 - Apple Support
08/13/24 About the security content of tvOS 17.4 - Apple Support
08/13/24 About the security content of tvOS 17.5 - Apple Support
08/13/24 About the security content of visionOS 1.2 - Apple Support
08/13/24 About the security content of watchOS 10.2 - Apple Support
08/13/24 About the security content of watchOS 10.4 - Apple Support
08/13/24 About the security content of watchOS 10.5 - Apple Support
08/13/24 About the security content of Xcode 15.3 - Apple Support

This is interesting, but what’s the idea? Parsing those specific URLs would only give you information about particular releases.

Note that if it’s just releases you’re after then Apple already have a page for all new releases, including the betas. You can follow it with RSS, too.

For security announcements, their mailing list is still working and they send PGP-signed email to it.
https://lists.apple.com/mailman/listinfo/security-announce/

My experience with security updates is that sometimes there are small updates that aren’t important in any way and other times Apple adds in CVE entries that it didn’t want to publicize for some reason initially. Diffing is the only way to know.

As with most programming things, everything is doable, usually comes down to a less frequently asked question of if it should be done. Seems like there’s a few different needs here.

The notification of changes seems a very common need and as others have stated, clearly best served by RSS or similar. Ideally this would come from Apple since it would be automatic when updating their pages. Might even be worth a feature request or bug report, perhaps on their support forums? Technically not a complicated feature, even without knowing their CMS.

To see what has actually changed, as @ace mentioned, requires a record of before/after and ability to diff. Legalities aside, the lazy developer in me cringes at the idea of duplicating all the pages and their versions. I agree it’s probably the only way you’d ever get that feature as Apple would never add it, but I just hate any unnecessary data duplication and syncing, it’s never as simple as it seems.

That being said, with some of the ideas posted earlier, it would be quite easy to use the URL IDs as uniqueIDs, store each version as a text blob, and then display the differences. It would probably be best to separate out the main content from all the html, simply to minimize the diffs to the most important information. One question never answered is if this is just an English language resource, or do we need to save the pages for every language?

Honestly the simplest way to get 95% of the features you’d want would be to use git since the problem you are talking about is versioning, with existing solutions. A public repository on GitHub would give you backups, diffing, versions, etc all built in. Using some of the ideas mentioned above, anytime a page is modified, simply download the raw URL, extract the main content div to the appropriate file, and do a git commit.

While the interface isn’t as “consumer level” friendly for diffing, it’s familiar to many, and as your suggesting this geared towards professionals, would be good enough. Honestly not sure how much easier you could make it other than making the diff feature more prominent than it is on GitHub; maybe making a shell site for the files that links directly to the diff view of any article.

All of that just brings me back to the irony that Apple is assuredly using some kind of versioning system for all those resources, so the entire thing would be duplication unfortunately.

As a side note Apple does have a similar feature on its developer site, and some of its support pages. It’s not a full diff, but it is exposing versions of the pages for when software updates change the interface. So the functionality exists.

One example, note the selectors at the top for the language (swift vs Objective-C) and API changes.

On the support side it took me a while to find an example of what I remembered, and I’m wondering if it’s only in the “guide” pages? But there are pages that have different OS versions in a drop down, that must be pulling different versions of content as it’s been updated.

1 Like

Storing revisions so they can be compared is next on the to-do list.

I love hearing people kibbitz about this while others are fiddling with code in the background. :-)

1 Like

Since we’re entertaining Adam, interestingly there seems to be a small faux pax in the published date tag on apple’s support pages. The raw HTML:

<div class='mod-date'>
     <span>Published Date:</span>&nbsp;<time dateTime=April 26, 2024itemprop='datePublished'>April 26, 2024</time>
</div>

While not well defined, the convention seems to be that the dateTime property inside the time tag is usually in a machine readable format (ex “2024-04-26”). More significantly, no space before the beginning of the itemprop key makes parsing that questionable, and ideally the value for the dateTime would be encased in quotes, esp if using human readable format (ex: “April”).

Only checked a sample size of 2, but probably the same across all pages.

2 Likes

Definitely malformed HTML. Shame on Apple.

See also:

Apple has apparently just changed the URL scheme to a simple six-digit numbering approach, with all the old HT URLs redirecting. From MacAdmins News:

1 Like

Another ASAT-fiddler here: my tool(s) are now able to retrieve, parse and store (in a local SQL-database) an(y??) number of different ranges of support notes, but before I am try my luck and take a look what comes when trying to download a range 100.000 to 999.999 …
Does anybody have any info which numbers now make sense to take a shot at?
Any suggestions? Or experiences what was successful?

Some very recent ones seem to be in the high 120000 range, so there is probably no need to go much higher than that in initial testing.

I’ve had very poor luck in guessing randomly at the low numbers in the new ranges.

As with some others here, I’m not a complete stranger to site-scraping scripts — though it’s been a little while since I’ve developed anything “big” enough to have to worry particularly about robots.txt files.

Still, I guess I have to wonder whether it’s really worth the time and effort. Wouldn’t a search-engine search on site:support.apple.com sorted by recency give you what you’re looking for quickly?

https://kagi.com/search?q=%22security+content%22+site%3Asupport.apple.com&order=2

for the security content of recent releases, for example, or

https://kagi.com/search?q=macOS+Sequoia+site%3Asupport.apple.com&order=2#

for macOS 15.

The search-engine sites are already doing the heavy lifting, it seems to me…

Hmm, in the big-data projects I was on that would be the exact reason why you would want to do a tool like this … the search engine/retrieval approach works fine if you are OK with actively doing a search and interpreting the results when you want to review the status.
Adams suggestion works when you want to tool tell/alert you “heh, look here there is something new” and this is what you might want to look at/consider.
In my experience it boils down to what do you want to have a tool like this do for you.

1 Like

No, because the goal is to learn what has changed, not search for something to see if it has changed.

Apple adds and changes information all the time, and I think lots of people would find that information helpful rather than having to stumble upon it later.

2 Likes