{ "componentChunkName": "component---src-templates-issues-tsx", "path": "/issues/263", "result": {"data":{"issuesJson":{"id":"914c9cd8-6baa-5e7f-97f7-8c60c75f37c9","title":"airwallex/k8s-pod-restart-info-collector: 监视 Pod 更改并在 Pod 重新启动时将 K8s Pod 重新启动原因、日志和事件收集到 Slack 通道","number":263,"bodyHTML":"

\n

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

Repos	airwallex/k8s-pod-restart-info-collector
Views	$\"views\"$
Stars	$\"stars\"$
Forks	$\"forks\"$
License	$\"license\"$
UpdatedAt	$\"last-commit\"$
CreatedAt	$\"create-at\"$

\n\n

\n

\n

\n

k8s-pod-restart-info-collector

\n

k8s-pod-restart-info-collector is a simple K8s customer controller that watches for Pods changes and collects K8s Pod restart reasons, logs, and events to Slack channel when a Pod restarts.

\n

For more information, see the blog on Medium: Automated Troubleshooting of Kubernetes (K8s) Pods Issues

\n

This project is actively used and maintained by Airwallex DevOps team.

\n

Overview of the Data Collected

\n

Here are two Slack screenshots of the example messages.

\n

Brief Alert Message

\n

\n

Detailed Alert Message

\n

As shown below, by clicking “Show more”, we can see the Reason, “Pod Status”, “Pod Events”, “Node Status and Events”, and “Pod Logs Before Restart”.

\n

\n

How to test and develop locally

\n

export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/xxxxx/xxxxx\ngo run .

\n

Install using Helm

\n

Replace the slackWebhookUrl, clusterName and slackChannel.

\n

helm upgrade --install k8s-pod-restart-info-collector ./helm \\\n   --set slackWebhookUrl=\"https://hooks.slack.com/services/Change-Me\" \\\n   --set clusterName=\"Change-Me\" \\\n   --set slackChannel=\"Change-Me\"

\n

Check Commands:

\n

# check commands\nkubectl get pod,deploy,sa,secret -l app.kubernetes.io/instance=k8s-pod-restart-info-collector\nhelm status k8s-pod-restart-info-collector\nhelm get values k8s-pod-restart-info-collector\nhelm get manifest k8s-pod-restart-info-collector\nhelm get all k8s-pod-restart-info-collector\n# see logs\nkubectl logs deployment/k8s-pod-restart-info-collector -f

\n

Run a debug-pod to verify the collector:

\n

kubectl run debug-pod --image=alpine -- date;sleep 30\nkubectl get pod debug-pod -w

\n

Uninstall

\n

To uninstall/delete the k8s-pod-restart-info-collector helm release:

\n

helm uninstall k8s-pod-restart-info-collector

\n

\n
The command removes all the Kubernetes components associated with the chart and deletes the release.
\n

\n

Helm Parameters

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

Name	Description	Value
`clusterName`	K8s cluster name (Display on slack message)	required
`slackUsername`	Slack username (Display on slack message)	default: `\"k8s-pod-restart-info-collector\"`
`slackChannel`	Slack channel name	default: `\"restart-info-nonprod\"`
`muteSeconds`	The time to mute duplicate pod alerts	default: `\"600\"`
`ignoreRestartCount`	The number of pod restart count to ignore	default: `\"30\"`
`ignoredNamespaces`	A set of namespaces to be ignored. This should be provided as a comma-separated list or a regular expression.	default: `\"\"`
`ignoredPodNamePrefixes`	A set of pod name prefixes to be ignored. This should be provided as a comma-separated list or a regular expression.	default: `\"\"`
`watchedNamespaces`	A set of namespaces to be watched. This should be provided as a comma-separated list or a regular expression.	default: `\"\"`
`watchedPodNamePrefixes`	A set of pod name prefixes to be watched. This should be provided as a comma-separated list or a regular expression.	default: `\"\"`
`ignoreRestartsWithExitCodeZero`	Whether restart events with an exit code of 0 should be ignored	default: `false`
`slackWebhookUrl`	Slack webhook URL	required if slackWebhooUrlSecretKeyRef is not present
`slackWebhookurlSecretKeyRef.key`	Slack webhook URL SecretKeyRef.key
`slackWebhookurlSecretKeyRef.name`	Slack webhook URL SecretKeyRef.name

\n

FAQ

\n

\n
When will the collector send Pod restart messages to Slack channel?
\n
When a Pod restarts. However, if one of the following conditions is met, the messages are not sent.
\n
1. Pod restartCount > 30
2. In the previous 10 minutes, the same Pod restart message was sent
\n
\n
How to customize slack channel for each pods
\n
Adding alert-slack-channel: \"your-slack-channel-name\" to Pod annotations or labels.
\nFor example, a label: alert-slack-channel: \"restart-info-nonprod\"
\n

\n

How to write a K8s controller

\n

Please refer to:

\n

\n

Copyright and license

\n

Copyright [2022] [Airwallex (Hong Kong) Limited]

\n

Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

\n

http://www.apache.org/licenses/LICENSE-2.0

\n

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

\n

Contribution

\n

If you are interested in contributing, see CONTRIBUTION.md.

","updatedAt":"2025-05-21T16:04:22Z","upvoteCount":null,"author":{"login":"eryajf","avatarUrl":"https://avatars.githubusercontent.com/u/33259379?u=e4a4090a38ac2473aaed4ef9945233636776c6c3&v=4","url":"https://github.com/eryajf"},"category":null,"labels":{"edges":[{"node":{"name":"更多","color":"25B472"}},{"node":{"name":"K8S-Tools","color":"2d411a"}},{"node":{"name":"airwallex","color":"81b950"}}]},"comments":{"edges":[]}}},"pageContext":{"number":263,"previous":{"title":"ben1234560/k8s_PaaS: 基于Kubernetes一步步部署成PaaS/DevOps的教程文档","number":262},"next":{"title":"chaseSpace/k8s-tutorial-cn: 作者整理的中文 k8s 教程，适合初学者阅读","number":264}}}, "staticQueryHashes": ["151096407","2861350382"]}