[Stack Monitoring] Deprecate internal collection topics

[ycombinator]

We now have the ability to monitor Elasticsearch, Kibana, Logstash, Beats, and APM Server with Metricbeat. The previous method of monitoring stack products, using collection code internal to each product that shipped monitoring data to a custom Monitoring Bulk API endpoint, is now deprecated.

We should reflect this deprecation in documentation as well, specifically in these areas:

• Stack docs: https://www.elastic.co/guide/en/elastic-stack-overview/current/xpack-monitoring.html
• Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/monitor-elasticsearch-cluster.html
• Kibana: https://www.elastic.co/guide/en/kibana/current/configuring-monitoring.html
• Beats: https://www.elastic.co/guide/en/beats/heartbeat/current/monitoring.html (Using Heartbeat as a sample here, but applicable to all Beats and APM Server documentation)
• Logstash: TBD; PR introducing Metricbeat collection is currently WIP: [Doc]Add metricbeat as monitoring option logstash#11033

[cachedout]

Hi @lcawl. I've started some of this:

Beats: elastic/beats#14887
Elasticsearch: elastic/elasticsearch#49758

[cachedout]

Elasticsearch done: elastic/elasticsearch#49758

[insukcho]

Just want to make sure that Beat also has the same policy. So, we need to use separate Metricbeat to monitor Metricbeat, right?

If yes, we should change the documentation content because users could be misleading that we recommend using deprecated internal collection:

The benefit of using internal collection instead of Metricbeat is that you have fewer pieces of software to install and maintain.