How to Report Bugs to Apple So They Get Fixed

8 posts were merged into an existing topic: External disk spontaneously dismounting

The purpose of this category is to determine whether or not the issue is due to a bug introduced by Apple in the beta or a problem that the developer needs to adapt to. If the former, then Apple should try to fix there bug. If it’s the result of something that Apple intentionally changed or deprecated, then they may get with the developer to let them know, but that’s a big MAY. I suspect that if the developer is a big enough partner such as Microsoft or Adobe then there will be some communication between the two, since the bigger developers generally are already running betas and they have corporate engineering representatives that routinely communicate such things. None of the small developers that I know have ever told me that they received such communications from Apple.

As far as beta testers are concerned, the NDA does not allow those users to contact developers about issues they have with macOS betas, which is designed to give Apple the first opportunity to fix any issue that is their bug so that developers don’t waste time trying to fix something that will be resolved before macOS release. But I’ve seen that restriction widely violated by testers. I’ve also know Apple to occasionally direct testers to contact the developer about an issue.

1 Like

Thanks for the answer.

Can you please describe what happens to CRASH REPORTS that are generated for all crashes, not just public beta testers?

Does anyone actually read them! Are they routinely also coied to the developer? Are they tallied or countedsoan app that crashes 100 times gets moreattention that an app that crashes 5 times?

Is all that information about the different threads and the list of instructions and what crrashed of practical use to the developer? They are gibberish to me.

You might find this interesting:

Technical Note TN2151: Understanding and Analyzing Application Crash Reports.

Developers can get crash reports. As @das wrote, Apple collects them on a server. Developers are notified about new crash reports and can download them for analysis.

When used in conjunction with an application’s source code (originally used to build the app) and its debug information (generated by the compiler alongside the app), the crash report includes a list of system exceptions (describing why the OS killed the app) and a stack trace for every thread in the application at the time it crashed. It also includes the contents of the CPU registers for every thread.

This information is all critical information for any developer trying to identify and fix a non-trivial problem.

The only thing more useful would be a full core dump containing the complete contents of the app’s memory at the time of the crash. Core dumps can allow the developer to perform post-mortem debugging, where they re-create the app’s (crashed) state in memory so debugging tools can be used to view it.

Apple doesn’t provide core dumps with crash reports because they can be very big (potentially gigabytes of data) and there can be serious privacy issues (since they contain all of the app’s data, including stuff you may not want the developers to know about).

1 Like

In this case, your best bet is to call Apple support and work through them. Reporting bugs is only useful if you (and thus Apple) can reproduce the problem at will. Particularly in a situation like this, there are simply too many variables for an engineer to be able to track down the problem.

Crash reports are generated automatically, if you agree to send analytics info to Apple when you set up your device.

Crash reports are taken very seriously at Apple. Crash reports for Apple software is assigned to the team that owns that software. There’s a list of the top crashing bugs for each team, and they’re expected to fix them. Crashes that happen more often are ranked higher.

As @Shamino said, crash reports for 3rd party apps are made available to developers in their developer account, it’s up to developers to read those and fix them.

All that gibberish really is important info, and very helpful in tracking down a crash. The tech note @Shamino linked to has a good explanation of what’s in a crash report, if you want to learn about it.

2 Likes

Although this is certainly true for regular users, Apple support (both AppleCare and Genius Bars) are instructed not to deal with beta OS issues. That includes troubleshoot a Mac with a beta OS installed. I’m sure there have been exceptions, but that is what has been put out.

As a Catalina beta tester, I filed this bug. Apple said there were more than 10 similar reports.

It has gotten a lot better, I only see the error occasionally so it might just be physical movement of the device.

But it used to generate that Not Ejected Properly error repeatedly just sitting on the desk and not being used.

My suggestion is to update to the latest release version of Catalina. Catalina is very stable at this point.

Work has shifted to Big Sur although there was a release of another Catalina beta build for Mac OS and iOS 13 just last week.

So maybe they are still working on some aspects of Catalina for a final public release. I am sure they want to make sure the transition to Big Sur is flawless.

Sierra to High Sierra release (not beta) on V1 update bricked a number of 2012 MacBook Pros. Mine included. The 2012 MBP was compatible with everything in High Sierra except the new file structure.

When the updater tried to rewrite the drive, the result was a bricked machine. Apple had to give those affected brand new free replacement machines direct from China in exchange for the bricked machine.

And V1.1 of the updater was released within 72 hours. I would love to hear the true inside story of that screw-up and if anyone got fired over such an expensive mistake.

I am holding off on testing Big Sur until the release notes are down to one or two lines. And I think I will load it on an external drive once I learn how. Because the beta build Apple sends doesn’t seem to allow choice of drive.

Maybe boot from an external drive with Catalina and it will update the startup external drive to Big Sur?

Howard Oakley just wrote this interesting piece basically encouraging most users to stop reporting bugs to Apple. He claims we’re enabling Bad Apple and urges us to stop in order to force the company to finally get serious about testing and QC.

I saw that, and I can understand the frustration, but it doesn’t feel to me like Apple QA would then devote more resources to looking for bugs on their own. It would just mean fewer bugs reported and fewer bugs fixed, which doesn’t help anyone.

Well his take on that is to instead devote time/effort to shame Apple publicly. That he claims is the one thing Apple reacts to and he believes that will eventually get their QA in line.

The way I see it we have been unsuccessfully doing the whole testing and reporting thing for the past decade or so while we watched reliability steadily decline (“it just works” has started becoming a faint memory) and the cadence between update and update-to-fix-the-last-update became shorter and shorter. Well, if you keep doing the same thing but the outcome doesn’t improve, to me that’s usually a sign it’s time to change strategy. I therefore hesitate to dismiss what he says.

1 Like

I’ve done more than my share of shaming Apple publicly over the years, and my experience is that unless something is significant enough to get picked up across the Apple media ecosystem, it gets ignored. So that’s fine for something like the free space in the Big Sur installer problem, but it won’t work for bugs that are smaller or less serious. Which is not to say that reporting bugs works reliably either, but at least they’re in the system for Apple to fix, which won’t be true otherwise.

I remember reading a column from decades ago where the writer mused about the perception that Apple released better quality software (as in fewer bugs) than Microsoft did. He noted that Apple didn’t release bug-free software or even software with fewer bugs, but it released software with the “right” bugs. Back then, Apple made the effort to track down and fix even minor bugs that would detract from the user’s experience. Nowadays, at least to me, that doesn’t seem to be the case.

I think there needs to be a change in the internal culture of Apple before we’ll see any significant increase in quality. It doesn’t matter if we report bugs or not, if all that happens is Apple engineers look at a bug report and routinely decide it’s not worth spending the time to fix it.

3 Likes

Alas, there’s no way to know if the grass really was greener in the past, given that we seldom hear about the bugs Apple is undoubtedly fixing behind the scenes all the time. And, of course, today’s systems are orders of magnitude more complex. :-)

The iOS 14/Big Sur cycle of releases seems significantly more bug-free than the iOS 13/Catalina cycle, which isn’t surprising given how much flak the company took last year. It’s worth re-reading @das’s 2019 article on that topic and then returning to the one whose comment thread we’re in:

Let’s not confuse the improvement from iOS 13->14/Catalina->Big Sur with the overarching decade-long trend towards reduced reliability and increasingly buggy behavior. “It just works” is no longer the motto. But many of us still remember the times when that was actually to large extent the case. To those who worked through both periods there is no confusing these two very different realities. And IMHO this is not the time for relativism if we expect to see things change for the better.

3 Likes

Well, if a trend is ever to reverse, it has to start reversing at some point, which would seem to be the case with iOS 14/Big Sur in comparison with the previous releases. And while there’s no question that there are bugs, some of which I even notice and report, it’s extremely uncommon for any of them to actually prevent me from getting my work done. My Mac and iPhone and iPad and Apple Watch and Apple TV and AirPods do just work, and the bugs are mostly annoyances.

Of course, lots of things in the wider world don’t always work the way I want, too. I just don’t get that exercised about things I can’t control.

You’re right. I should perhaps lay the blame on the process rather than the culture. With the much more complex nature of our devices these days, there are simply more ways for a user to encounter buggy behavior. Even if the bugs per lines of code (or whatever is the appropriate metric) has remained constant over the years, the bugs-encountered-per-user rate seems to have gone up noticeably.

@mjtsai also weighed in on this, noting that he can’t see any rhyme nor reason to why some bugs he reports are fixed and others aren’t.

https://mjtsai.com/blog/2021/02/18/why-reporting-bugs-to-apple-may-harm-software-quality/

If a reported bug is critical, can we expect it to be fixed in 10 days from being reported, and released in the just immediately next beta?
Or does a bug get fixed and released only after around 2 months?

You and I have absolutely no way of knowing. Apple doesn’t publish their procedures for things like this.

We can assume, however, that they prioritize bugs based on how critical their engineers and managers consider them. The most critical bugs (security, bugs that affect system stability or data integrity) will get fixed pretty quckly - either as a part of the next system update or a special mid-update patch.

Other bugs will get either rolled into the next scheduled update or may be put on hold until the next major OS version (e.g. if it changes the user interface).