Sorry, but I’m even more confused now. I was able to use Feedback Assistant to report a bug in the current shipping 10.15.5. I’m not in any beta program. Did I just send a bunch of information to /dev/null? Should I instead have used the feedback website?
I wish I could answer that for you, but I have no idea how or if those are being handled nor even who at Apple would be able to answer, but thanks for verifying that a non-beta tester is able to at least prepare and send reports. I know that the various groups of beta test submissions are routed differently to the engineers responsible for managing them. As mentioned before, they are sorted and transcribed into a “radar” report (at least that was the system before the latest reorganization) so that the appropriate engineering section can take responsibility for it. You might get some feedback as to the status of your individual reports and how many similar reports have been submitted. You might get a request for additional information. That would tend to indicate that your reports are being handled. Just be aware that most submissions, even by beta testers, are never replied to.
Your experience isn’t unique. I don’t recall that I have ever heard that a public beta program participant ever heard anything back. They do get aggregated with similar reports, so I would suspect that they pay closer attention to the ones with the most reports. That doesn’t always insure that they will get fixed as they may be too hard or just receive a lower priority.
Thanks, Al. Appreciate the background. My Feedback Assistant lists recent similar reports as none and ‘resolution’ as none which I believe is the same it said right after I filed. So from all I can tell nothing has happened (so far).
Actually Al, I was just making a grammar nerd correction of the statement in the original article: “The Feedback Assistant app is installed in macOS 10.15 Catalina and later.” The implication of that statement, especially to someone who is new to macOS, is that it is an app that was introduced by macOS Catalina. I was merely pointing out that it has existed for quite some time. Whether or not it has any utility in previous versions of macOS was not stated nor meant to be implied. Hence the use of FWIW (For what it’s worth) at the beginning of my comment.
Having said that, I really appreciated your detailed and insightful explanations throughout the rest of the thread to date. Thank you for that background!
Great article. Thanks for sharing it.
I have a question on the process when I report a bug in a third party app running on a OS X public beta. One of the categories in Feedback Assistant is to report third party app bugs.
Am I wasting my time to report a bug in a third party app using Feedback Assistant?
Most developers it seems do not ever load a Mac OS beta. So if the beta OS causes bugs to appear in their app, the developers don’t care.
They assume the Mac OS bug will get fixed or the OS feature causing the bug in their app may not even appear in the Golden Master. So they don’t start to triage their app bugs until the crunch right before public release.
In other words, App Developers seem to only deal with bugs in the current Apple public OS release. In other words, developers are behind by a release.
So what happens to those Feedback Assistant reports that report issues that occur in a specific third party app while running a Mac OS Public Beta?
What happens to Crash Reports that relate to a crash in a third party app that are sent to Apple? If the app bug caused a Crash Report should I also report it in Feedback Assistant?
If a third party app crashes repeatedly, generating multiple crash reports, does Apple in any way work with the developer to fix the problem? Does Apple ever apply subtle pressure on App Developers to fix bugs in their app? Or are the Crash Reports simply forwarded to the app developer and forgotten?
I’m not a developer so I do not understand the process of third party app development and how closely Apple Engineers work with the Third Party App engineers.
The resolution of a bug also seems to depend on the “Which area are you seeing an issue with?” reported with the bug.
I have seen the same issue reported against a particular area get comments on Recent Similar Reports: Less than 10, while the same issue if reported against another area gets Recent Similar Reports: None.
Why does this happen.
This article was really useful… In my 30+ years of using Apple gear, I’ve never heard of this Feedback Assistant app. I always used Apple’s “Product Feedback” page which does not ask for any details. And who knows who reads the input from that link…
Correction, I did get a request from an Apple engineer once regarding a bug I submitted using “Product Feedback”. He asked for a Quicktime video showing how the bug occurs. I sent that in and followed up later to see. He responded and said the issue was being reviewed. Nice.
Feedback Assistant is relatively new. For years the Bug Reporter web site was how 3rd party developers to report bugs to Apple. There wasn’t really a single, coordinated way for users to report bugs. Apple does track calls to AppleCare and Genius Bar visits, to see what users are having trouble with.
The categories you see as outsider reporting a bug are not the same categories Apple uses in Radar. Radar has thousands of categories and subcategories, too many for outsiders to wade through, and many for secret, unreleased products.
If there’s no obvious category for the bug you’re seeing, you’ll pick one that seems close. The category in your bug report is mapped to a category in Radar, which is owned by an engineering team, which reads your bug report. But it might be mapped to the wrong team for that feature, there’s really no way for you to know. Hopefully the team that gets your bug will reassign it to the right team, but sometimes people get busy, and bug reports fall through the cracks.
If it’s not fixed in the next beta release, don’t be afraid to write a new bug report, and perhaps try a different area, if that seems applicable.
Unfortunately, I’d say reporting 3rd party bugs to Apple is probably a waste of time. I worked at Apple before Feedback Assistant, but AFAIK bugs reported in 3rd party apps were almost never passed on to those 3rd parties.
For small companies, I’d try contacting them directly. Often there’s just a couple of tech support people, who can pass concise, reproducible bug reports on to their engineers.
Crash reports are collected automatically by the OS, and are available for developers to read in their Apple Connect account. Many developers also use 3rd party crash reporting libraries.
I don’t think that’s true, most developers I know definitely test on beta OS releases, they want to be ready when the new OS ships. But Apple won’t let them ship updates for a beta OS, or even mention it in their release notes, because betas are still officially unreleased software. That’s why when the new OS finally ships, there’s a crush of developers trying to ship updates.
If it’s an important app like Photoshop, Apple will definitely alert them about problems. Sometimes Apple even changes the OS to work around the problem. But most 3rd party developers don’t get this personal attention.
8 posts were merged into an existing topic: External disk spontaneously dismounting
The purpose of this category is to determine whether or not the issue is due to a bug introduced by Apple in the beta or a problem that the developer needs to adapt to. If the former, then Apple should try to fix there bug. If it’s the result of something that Apple intentionally changed or deprecated, then they may get with the developer to let them know, but that’s a big MAY. I suspect that if the developer is a big enough partner such as Microsoft or Adobe then there will be some communication between the two, since the bigger developers generally are already running betas and they have corporate engineering representatives that routinely communicate such things. None of the small developers that I know have ever told me that they received such communications from Apple.
As far as beta testers are concerned, the NDA does not allow those users to contact developers about issues they have with macOS betas, which is designed to give Apple the first opportunity to fix any issue that is their bug so that developers don’t waste time trying to fix something that will be resolved before macOS release. But I’ve seen that restriction widely violated by testers. I’ve also know Apple to occasionally direct testers to contact the developer about an issue.
Thanks for the answer.
Can you please describe what happens to CRASH REPORTS that are generated for all crashes, not just public beta testers?
Does anyone actually read them! Are they routinely also coied to the developer? Are they tallied or countedsoan app that crashes 100 times gets moreattention that an app that crashes 5 times?
Is all that information about the different threads and the list of instructions and what crrashed of practical use to the developer? They are gibberish to me.
You might find this interesting:
Developers can get crash reports. As @das wrote, Apple collects them on a server. Developers are notified about new crash reports and can download them for analysis.
When used in conjunction with an application’s source code (originally used to build the app) and its debug information (generated by the compiler alongside the app), the crash report includes a list of system exceptions (describing why the OS killed the app) and a stack trace for every thread in the application at the time it crashed. It also includes the contents of the CPU registers for every thread.
This information is all critical information for any developer trying to identify and fix a non-trivial problem.
The only thing more useful would be a full core dump containing the complete contents of the app’s memory at the time of the crash. Core dumps can allow the developer to perform post-mortem debugging, where they re-create the app’s (crashed) state in memory so debugging tools can be used to view it.
Apple doesn’t provide core dumps with crash reports because they can be very big (potentially gigabytes of data) and there can be serious privacy issues (since they contain all of the app’s data, including stuff you may not want the developers to know about).
In this case, your best bet is to call Apple support and work through them. Reporting bugs is only useful if you (and thus Apple) can reproduce the problem at will. Particularly in a situation like this, there are simply too many variables for an engineer to be able to track down the problem.
Crash reports are generated automatically, if you agree to send analytics info to Apple when you set up your device.
Crash reports are taken very seriously at Apple. Crash reports for Apple software is assigned to the team that owns that software. There’s a list of the top crashing bugs for each team, and they’re expected to fix them. Crashes that happen more often are ranked higher.
As @Shamino said, crash reports for 3rd party apps are made available to developers in their developer account, it’s up to developers to read those and fix them.
All that gibberish really is important info, and very helpful in tracking down a crash. The tech note @Shamino linked to has a good explanation of what’s in a crash report, if you want to learn about it.
Although this is certainly true for regular users, Apple support (both AppleCare and Genius Bars) are instructed not to deal with beta OS issues. That includes troubleshoot a Mac with a beta OS installed. I’m sure there have been exceptions, but that is what has been put out.
As a Catalina beta tester, I filed this bug. Apple said there were more than 10 similar reports.
It has gotten a lot better, I only see the error occasionally so it might just be physical movement of the device.
But it used to generate that Not Ejected Properly error repeatedly just sitting on the desk and not being used.
My suggestion is to update to the latest release version of Catalina. Catalina is very stable at this point.
Work has shifted to Big Sur although there was a release of another Catalina beta build for Mac OS and iOS 13 just last week.
So maybe they are still working on some aspects of Catalina for a final public release. I am sure they want to make sure the transition to Big Sur is flawless.
Sierra to High Sierra release (not beta) on V1 update bricked a number of 2012 MacBook Pros. Mine included. The 2012 MBP was compatible with everything in High Sierra except the new file structure.
When the updater tried to rewrite the drive, the result was a bricked machine. Apple had to give those affected brand new free replacement machines direct from China in exchange for the bricked machine.
And V1.1 of the updater was released within 72 hours. I would love to hear the true inside story of that screw-up and if anyone got fired over such an expensive mistake.
I am holding off on testing Big Sur until the release notes are down to one or two lines. And I think I will load it on an external drive once I learn how. Because the beta build Apple sends doesn’t seem to allow choice of drive.
Maybe boot from an external drive with Catalina and it will update the startup external drive to Big Sur?
Howard Oakley just wrote this interesting piece basically encouraging most users to stop reporting bugs to Apple. He claims we’re enabling Bad Apple and urges us to stop in order to force the company to finally get serious about testing and QC.