Make zero_division parameter consistent in the different metric

[glemaitre]

This is an issue to report the step to actually take over the work of @marctorsoc in #23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the zero_division parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

• Introduce the zero_division parameter to the accuracy_score function when y_true and y_pred are empty.
  • ENH Add zero_division parameter for accuracy_score #29213
• Introduce the zero_division parameter to the class_likelihood_ratios and remove raise_warning.
• Introduce the zero_division parameter to the cohen_kappa_score function
  • ENH Add zero_division param to cohen_kappa_score #29210
• Introduce the zero_division parameter to the matthew_corr_coeff function
  • ENH add zero_division=nan for classification metrics #23183
  • FIX matthews_corrcoef returns zero for perfect prediction #28509
• Open a PR to make sure the empty input lead to np.nan in classification_report function.

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the zero_division parameter to the precision_recall_curve and roc_curve as well. This might add two additional items to the list above.

[glemaitre]

@StefanieSenger you might be interested in looking at some of the item.

[erikhuck]

+1 for this!

[Jaimin020]

I am working on task-1 (Introduce the zero_division parameter to the accuracy_score function when y_true and y_pred are empty.).

[erikhuck]

Thank you @Jaimin020 ! I needed that recently and had to write a wrapper of the accuracy function.

[StefanieSenger]

The intend of this issue is not quite clear to me. @glemaitre, could you clarify:

#23183 is about to be merged for the last 1,5 years, but is difficult to review. This issue is supposed to facilitate this process by splitting it into different pull-requests. However, it only lists tasks that would enhance the original PR (or add to it) and depend on it to be merged (except matthew_corr_coeff that had already been added to the old PR by your request).

1. How would adding those new PRs help maintainers to decide about the original PR?
2. Would those new PRs also be mergable if the original PR was in the end not adopted?

I can also see that the communication on in the original PR has been very frustrating for its author, because it was unclear, what he needs to deliver in order to push the reviewing process forwards. We should at least define this very clearly right now. This will help everybody involved (and everybody who wants to get involved) know what to expect and be efficient.

[StefanieSenger]

I realised that the solution lies in first splitting up the original PR as proposed by the author.

After that, the other tasks from the task list above can be tackled.

I will open PRs that split up the original PR during the next days.

[glemaitre]

I realised that the solution lies in first #23183 (comment) as proposed by the author.

Exactly.

I will open PRs that split up the original PR during the next days.

Do not hesitate to tackle a single PR first such that we make sure to get reviews.

[StefanieSenger]

Do not hesitate to tackle a single PR first such that we make sure to get reviews.

Good point. So I will start with this: "PR to add zero_division=np.nan to precision, recall, f1, fbeta_score, precision_recall_fscore_support and classification_report" (Edit: This was already done.)

I had missed the fact that the original PR had already mostly been adopted by #25531 and the remaining tasks are actually only those listed in the issue above. The original PR had now been closed by the author to avoid confusion (mine 🤣 ).

I will take care of the inividual PR for the cohen_kappa_score function.

[StefanieSenger]

I am also working on adding the zero_division param to class_likelihood_ratios().

[Jaimin020]

I have created a pull request for task-1 (#29213), but the 'codecov/patch' check is failing. Could someone please review it and advise me on how to pass this check?