Troubleshooting
This troubleshooting guide includes step-by-step instructions that should resolve most installation issues.
Troubleshooting Docker Standalone Installation of SigNoz
- Before you install SigNoz on Docker Standalone, ensure that all prerequisites are met.
- Run the
docker pscommand and ensure that thesignoz-clickhouseandsignoz-signozcontainers are running. If these containers are not running, increase the memory allocated to Docker. - If you don't see any data in the front-end, please wait two or three minutes.
- Run the
install.shscript again. The script will try to reinstall the containers that failed. - Follow the steps for uninstalling SigNoz section and then install SigNoz again by following the steps in the Install SigNoz on Docker Standalone section.
- If you're still facing issues trying to install SigNoz, please reach out to us on Slack
Troubleshooting ClickHouse container crashes
If the clickhouse container is unable to start due to certain reasons, it is recommended to take the following steps before recreating the container
- In the
deploy/dockerfolder, run the following command to see more detailed logs for the ClickHouse container:
sudo docker-compose ps -a
This will show you the status of all containers, including ClickHouse.
Verify system resources: Ensure your system has enough resources (CPU, memory, disk space) to run the ClickHouse container.
Check permissions: Make sure you have the necessary permissions to run Docker containers. Try running the installation script with sudo privileges.
Review Docker configuration: Ensure your Docker installation is up-to-date and properly configured.
- If the logs mention about
Permission deniederror, it is likely due to the correct permissions not set for the container. - Use the
chowncommand to change the ownership for the files that clickhouse container is unable to access. - If neither of this works, run the
install.shscript again as the root user or by usingsudoas a prefix.
SigNoz Otel Collector Address Grid
You might have specific set up for your application and SigNoz cluster. It might not be very clear on which address to use to send data to SigNoz.
Here is the SigNoz Otel Collector address grid which could be helpful:
| Where SigNoz is installed? | |||||
|---|---|---|---|---|---|
| VM (Docker) - Same Machine | VM (Docker) - Different Machine | K8s (Same Cluster) | K8s (Different Cluster) | ||
| Where your application is running? | VM (native/binary) | localhost:4317 | <otelcollector-IP>:4317 | <k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317 | <k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317 |
| VM (Docker) | 172.17.0.1:4317, otel-collector:4317(shared network) | <otelcollector-IP>:4317 | <k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317 | <k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317 | |
| Kubernetes | <otelcollector-IP>:4317 | <otelcollector-IP>:4317 | <release-name>-signoz-otel-collector.<namespace>.svc.cluster.local:4317 | <k8s-node-IP>:<otelcollector-node-port>, <k8s-loadbalancer-IP>:4317 | |
- For the
<otelcollector-IP>, use private IP address if the VM is in same private network. - Replace
<namespace>with SigNoz namespace and<release-name>with SigNoz helm release name. - In the case of k8s where the application and SigNoz are running in different k8s cluster, you will have to expose otel collector service. Set the service type to either
NodePortorLoadBalancer.
helm upgrade --install -n platform my-release signoz/signoz \
--set otelCollector.serviceType="<NodePort or LoadBalancer>"
Container signoz-init-clickhouse Hangs Forever
After installation, you might see sometime the signoz-init-clickhouse being stuck and from the logs it's shows:
docker logs signoz-init-clickhouse
# Fetching histogram-binary for linux/amd64
# Connecting to github.com (20.205.243.166:443)
This indicates the helper binary wasn’t fetched from GitHub. For these issues you should, restart the SigNoz containers and allow time for the helper binary to download:
docker compose restart
Last updated: June 6, 2024
