The resolution of a bug also seems to depend on the “Which area are you seeing an issue with?” reported with the bug.
I have seen the same issue reported against a particular area get comments on Recent Similar Reports: Less than 10, while the same issue if reported against another area gets Recent Similar Reports: None.
This article was really useful… In my 30+ years of using Apple gear, I’ve never heard of this Feedback Assistant app. I always used Apple’s “Product Feedback” page which does not ask for any details. And who knows who reads the input from that link…
Correction, I did get a request from an Apple engineer once regarding a bug I submitted using “Product Feedback”. He asked for a Quicktime video showing how the bug occurs. I sent that in and followed up later to see. He responded and said the issue was being reviewed. Nice.
Feedback Assistant is relatively new. For years the Bug Reporter web site was how 3rd party developers to report bugs to Apple. There wasn’t really a single, coordinated way for users to report bugs. Apple does track calls to AppleCare and Genius Bar visits, to see what users are having trouble with.
The categories you see as outsider reporting a bug are not the same categories Apple uses in Radar. Radar has thousands of categories and subcategories, too many for outsiders to wade through, and many for secret, unreleased products.
If there’s no obvious category for the bug you’re seeing, you’ll pick one that seems close. The category in your bug report is mapped to a category in Radar, which is owned by an engineering team, which reads your bug report. But it might be mapped to the wrong team for that feature, there’s really no way for you to know. Hopefully the team that gets your bug will reassign it to the right team, but sometimes people get busy, and bug reports fall through the cracks.
If it’s not fixed in the next beta release, don’t be afraid to write a new bug report, and perhaps try a different area, if that seems applicable.
Unfortunately, I’d say reporting 3rd party bugs to Apple is probably a waste of time. I worked at Apple before Feedback Assistant, but AFAIK bugs reported in 3rd party apps were almost never passed on to those 3rd parties.
For small companies, I’d try contacting them directly. Often there’s just a couple of tech support people, who can pass concise, reproducible bug reports on to their engineers.
Crash reports are collected automatically by the OS, and are available for developers to read in their Apple Connect account. Many developers also use 3rd party crash reporting libraries.
I don’t think that’s true, most developers I know definitely test on beta OS releases, they want to be ready when the new OS ships. But Apple won’t let them ship updates for a beta OS, or even mention it in their release notes, because betas are still officially unreleased software. That’s why when the new OS finally ships, there’s a crush of developers trying to ship updates.
If it’s an important app like Photoshop, Apple will definitely alert them about problems. Sometimes Apple even changes the OS to work around the problem. But most 3rd party developers don’t get this personal attention.
The purpose of this category is to determine whether or not the issue is due to a bug introduced by Apple in the beta or a problem that the developer needs to adapt to. If the former, then Apple should try to fix there bug. If it’s the result of something that Apple intentionally changed or deprecated, then they may get with the developer to let them know, but that’s a big MAY. I suspect that if the developer is a big enough partner such as Microsoft or Adobe then there will be some communication between the two, since the bigger developers generally are already running betas and they have corporate engineering representatives that routinely communicate such things. None of the small developers that I know have ever told me that they received such communications from Apple.
As far as beta testers are concerned, the NDA does not allow those users to contact developers about issues they have with macOS betas, which is designed to give Apple the first opportunity to fix any issue that is their bug so that developers don’t waste time trying to fix something that will be resolved before macOS release. But I’ve seen that restriction widely violated by testers. I’ve also know Apple to occasionally direct testers to contact the developer about an issue.
Developers can get crash reports. As @das wrote, Apple collects them on a server. Developers are notified about new crash reports and can download them for analysis.
When used in conjunction with an application’s source code (originally used to build the app) and its debug information (generated by the compiler alongside the app), the crash report includes a list of system exceptions (describing why the OS killed the app) and a stack trace for every thread in the application at the time it crashed. It also includes the contents of the CPU registers for every thread.
This information is all critical information for any developer trying to identify and fix a non-trivial problem.
The only thing more useful would be a full core dump containing the complete contents of the app’s memory at the time of the crash. Core dumps can allow the developer to perform post-mortem debugging, where they re-create the app’s (crashed) state in memory so debugging tools can be used to view it.
Apple doesn’t provide core dumps with crash reports because they can be very big (potentially gigabytes of data) and there can be serious privacy issues (since they contain all of the app’s data, including stuff you may not want the developers to know about).
In this case, your best bet is to call Apple support and work through them. Reporting bugs is only useful if you (and thus Apple) can reproduce the problem at will. Particularly in a situation like this, there are simply too many variables for an engineer to be able to track down the problem.
Crash reports are generated automatically, if you agree to send analytics info to Apple when you set up your device.
Crash reports are taken very seriously at Apple. Crash reports for Apple software is assigned to the team that owns that software. There’s a list of the top crashing bugs for each team, and they’re expected to fix them. Crashes that happen more often are ranked higher.
As @Shamino said, crash reports for 3rd party apps are made available to developers in their developer account, it’s up to developers to read those and fix them.
All that gibberish really is important info, and very helpful in tracking down a crash. The tech note @Shamino linked to has a good explanation of what’s in a crash report, if you want to learn about it.
Although this is certainly true for regular users, Apple support (both AppleCare and Genius Bars) are instructed not to deal with beta OS issues. That includes troubleshoot a Mac with a beta OS installed. I’m sure there have been exceptions, but that is what has been put out.
As a Catalina beta tester, I filed this bug. Apple said there were more than 10 similar reports.
It has gotten a lot better, I only see the error occasionally so it might just be physical movement of the device.
But it used to generate that Not Ejected Properly error repeatedly just sitting on the desk and not being used.
My suggestion is to update to the latest release version of Catalina. Catalina is very stable at this point.
Work has shifted to Big Sur although there was a release of another Catalina beta build for Mac OS and iOS 13 just last week.
So maybe they are still working on some aspects of Catalina for a final public release. I am sure they want to make sure the transition to Big Sur is flawless.
Sierra to High Sierra release (not beta) on V1 update bricked a number of 2012 MacBook Pros. Mine included. The 2012 MBP was compatible with everything in High Sierra except the new file structure.
When the updater tried to rewrite the drive, the result was a bricked machine. Apple had to give those affected brand new free replacement machines direct from China in exchange for the bricked machine.
And V1.1 of the updater was released within 72 hours. I would love to hear the true inside story of that screw-up and if anyone got fired over such an expensive mistake.
I am holding off on testing Big Sur until the release notes are down to one or two lines. And I think I will load it on an external drive once I learn how. Because the beta build Apple sends doesn’t seem to allow choice of drive.
Maybe boot from an external drive with Catalina and it will update the startup external drive to Big Sur?
Howard Oakley just wrote this interesting piece basically encouraging most users to stop reporting bugs to Apple. He claims we’re enabling Bad Apple and urges us to stop in order to force the company to finally get serious about testing and QC.
I saw that, and I can understand the frustration, but it doesn’t feel to me like Apple QA would then devote more resources to looking for bugs on their own. It would just mean fewer bugs reported and fewer bugs fixed, which doesn’t help anyone.
Well his take on that is to instead devote time/effort to shame Apple publicly. That he claims is the one thing Apple reacts to and he believes that will eventually get their QA in line.
The way I see it we have been unsuccessfully doing the whole testing and reporting thing for the past decade or so while we watched reliability steadily decline (“it just works” has started becoming a faint memory) and the cadence between update and update-to-fix-the-last-update became shorter and shorter. Well, if you keep doing the same thing but the outcome doesn’t improve, to me that’s usually a sign it’s time to change strategy. I therefore hesitate to dismiss what he says.
I’ve done more than my share of shaming Apple publicly over the years, and my experience is that unless something is significant enough to get picked up across the Apple media ecosystem, it gets ignored. So that’s fine for something like the free space in the Big Sur installer problem, but it won’t work for bugs that are smaller or less serious. Which is not to say that reporting bugs works reliably either, but at least they’re in the system for Apple to fix, which won’t be true otherwise.
I remember reading a column from decades ago where the writer mused about the perception that Apple released better quality software (as in fewer bugs) than Microsoft did. He noted that Apple didn’t release bug-free software or even software with fewer bugs, but it released software with the “right” bugs. Back then, Apple made the effort to track down and fix even minor bugs that would detract from the user’s experience. Nowadays, at least to me, that doesn’t seem to be the case.
I think there needs to be a change in the internal culture of Apple before we’ll see any significant increase in quality. It doesn’t matter if we report bugs or not, if all that happens is Apple engineers look at a bug report and routinely decide it’s not worth spending the time to fix it.
Alas, there’s no way to know if the grass really was greener in the past, given that we seldom hear about the bugs Apple is undoubtedly fixing behind the scenes all the time. And, of course, today’s systems are orders of magnitude more complex.
The iOS 14/Big Sur cycle of releases seems significantly more bug-free than the iOS 13/Catalina cycle, which isn’t surprising given how much flak the company took last year. It’s worth re-reading @das’s 2019 article on that topic and then returning to the one whose comment thread we’re in:
Let’s not confuse the improvement from iOS 13->14/Catalina->Big Sur with the overarching decade-long trend towards reduced reliability and increasingly buggy behavior. “It just works” is no longer the motto. But many of us still remember the times when that was actually to large extent the case. To those who worked through both periods there is no confusing these two very different realities. And IMHO this is not the time for relativism if we expect to see things change for the better.