{ "componentChunkName": "component---src-templates-issues-tsx", "path": "/issues/215", "result": {"data":{"issuesJson":{"id":"75022916-52b7-527c-ba5b-f83a71390586","title":"madneal/gshark: 根据关键字等规则，从 GitHub、GitLab 等扫描代码泄露、敏感信息泄露等安全问题","number":215,"bodyHTML":"

\n

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n

Repos	madneal/gshark
Views	$\"views\"$
Stars	$\"stars\"$
Forks	$\"forks\"$
License	$\"license\"$
UpdatedAt	$\"last-commit\"$
CreatedAt	$\"create-at\"$

\n\n

\n

\n

\n

\n $\"GShark$ \n

GShark

\n

Scan for sensitive information easily and effectively.

\n

\n

GShark $\"Go$ $\"Release\"$

\n

The project is based on Go and Vue to build a management system for sensitive information detection. For the full introduction, please refer to articles and videos. For now, all the scans are only targeted to the public environments, not local environments.

\n

For the usage of GShark, please refer to the wiki.

\n

Key Features

\n

🌐 Multi-Platform Support: GitHub, GitLab, Searchcode, Postman, and more*
🔍 Flexible Rule Management: Custom scanning rules and filtering with whitelist/blacklist support*
🔑 Fine-grained Access Control: Configurable menu and API permissions
🔄 Subdomain Discovery: Integrated gobuster for subdomain enumeration
🚀 Docker Deployment: Containerized deployment for easy setup
📊 Visual Management Interface: Intuitive web interface for task and result management

\n

Quick start

\n

Docker Deployment

\n

# Clone the repository\ngit clone https://github.com/madneal/gshark\n\ncd gshark\n\n# Build and start the container\ndocker-compose build && docker-compose up \n

\n

Important

Before the MySQL database initial, the scanner container will exit. Need to restart the scanner after the MySQL database initial.

\n

\n

Manual Deployment

\n

Requirements

\n

Nginx
MySQL(version above 8.0)

\n

It is recommended to deploy the Front-End project using Nginx. Place the dist folder in /var/www/html, and adjust the nginx.conf file (/etc/nginx/nginx.conf for Linux) to set up reverse proxy for the backend service. For detailed deployment tutorials, you can watch videos on bilibili or youtube. For deployment on Windows, refer to this link.

\n

Nginx

\n

Can use nginx -t to locate the nginx.conf file, then modify the nginx.conf:

\n

// config the user accoring to your need\nuser  www www;\nworker_processes  1;\n\nevents {\n    worker_connections  1024;\n}\n\nhttp {\n    include       mime.types;\n    default_type  application/octet-stream;\n    sendfile        on;\n    keepalive_timeout  65;\n    server {\n        listen       8080;\n        server_name  localhost;\n\n        location / {\n            autoindex on;\n            root   html;\n            index  index.html index.htm;\n        }\n        location /api/ {\n            proxy_set_header Host $http_host;\n            proxy_set_header X-Real-IP $remote_addr;\n            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n            proxy_set_header X-Forwarded-Proto $scheme;\n            rewrite ^/api/(.*)$ /$1 break;\n            proxy_pass http://127.0.0.1:8888;\n        }\n        error_page   500 502 503 504  /50x.html;\n        location = /50x.html {\n            root   html;\n        }\n    }\n    include servers/*;\n}\n\n

\n

The deployment work is straightforward. Find the corresponding version zip file from releases.

\n

Unzip and copy the files inside dist to /var/www/html folder of Nginx.

\n

unzip gshark*.zip\ncd gshark*\nmv dist/* /var/www/html/\n# for Mac\nmv dist/* /usr/local/www/html/\n

\n

Start the Nginx and the Front-End is deployed successfully.

\n

Tip

If you installed Nginx by Homebrew, you need to stop Nginx by:

\n

brew services stop nginx

\n

Start Nginx for Ubuntu:

\n

systemctl start nginx

\n

\n

Server service

\n

./gshark serve

\n

Initially, rename config-temp.yaml to config.yaml. After that, you can run the gshark binary file directly. Then, access localhost:8080 for local deployment.

\n

If you haven't initialized the database before, you will be redirected to the database initialization page first.

\n $\"image\"$ \n

Scan service

\n

./gshark scan

\n

For the scan service, it's necessary to config the corresponding rules. For example, GitHub or Gitlab rules.

\n

Incremental Deployment

\n

For the incremental deployment, sql.md should be executed for the corresponding database operations.

\n

Development

\n

Server

\n

git clone https://github.com/madneal/gshark.git\ncd server\ngo mod tidy\nmv config-temp.yaml config.yaml\ngo build

\n

Run the web server:

\n

go build\n./gshark serve

\n

Or

\n

go run main.go serve

\n

Run the scan task:

\n

go build\n./gshark scan

\n

Or

\n

go run main.go scan

\n

Web

\n

cd ../web\n\nnpm install\n\nnpm run serve\n

\n

Usage

\n

Add Token

\n

GitHub

\n

To execute the scan task for GitHub, you need to add a GitHub token for crawl information in GitHub. You can generate a token in tokens. Most access scopes are enough. For the GitLab search, remember to add a token too.

\n

$\"iR2TMt.md.png\"$

\n

Postman

\n

Obtain the postman.sid cookie:

\n $\"image\"$ \n

Rule Configuration

\n

For the Github or Gitlab rule, the rule will be matched by the syntax in the corresponding platforms. Directly, you config what you search at GitHub. You can download the rule import template CSV file, then batch import rules.

\n

\n

Filter Configuration

\n

Filter is only addressed to GitHub search now. There are three classes of filters, including extension, keyword, sec_keyword. For extension and keyword, they can used for blacklist or whitelist.

\n

For more information, you can refer to this video.

\n

Configuration

\n

You are supposed to rename config-temp.yaml to config.yaml and config the database information and other information according to your environment.

\n

GitLab Base Url

\n

\n

FAQ

\n

Default username and password to login

\n

gshark/gshark

\n

Database initial failed

\n

make sure the version of MySQL is over 5.6. And remove the database before initialing the second time.

\n

go get ./... connection error

\n

It's suggested to enable GOPROXY(refer this article for golang upgrade):

\n

go env -w GOPROXY=https://goproxy.cn,direct\ngo env -w GO111MODULE=on\n

\n

When deploying the web to nginx, the page was empty

\n

try to clear the LocalStorage

\n

Resources

\n

Articles

\n

\n

Videos

\n

\n

License

\n

Apache License 2.0

\n

404StarLink 2.0 - Galaxy

\n

\n

GShark 是 404Team 星链计划2.0中的一环，如果对 GShark 有任何疑问又或是想要找小伙伴交流，可以参考星链计划的加群方式。

\n

https://github.com/knownsec/404StarLink2.0-Galaxy#community

","updatedAt":"2025-05-21T16:01:00Z","upvoteCount":null,"author":{"login":"eryajf","avatarUrl":"https://avatars.githubusercontent.com/u/33259379?u=e4a4090a38ac2473aaed4ef9945233636776c6c3&v=4","url":"https://github.com/eryajf"},"category":null,"labels":{"edges":[{"node":{"name":"更多","color":"25B472"}},{"node":{"name":"安全扫描","color":"bf5886"}},{"node":{"name":"madneal","color":"c69e2e"}}]},"comments":{"edges":[]}}},"pageContext":{"number":215,"previous":{"title":"snowdreamtech/gserver: 使用 gin 和 golang 构建的简单静态 HTTP 服务器。跨平台，零配置","number":214},"next":{"title":"moovweb/gvm: 一个简单的工具，用于多个 Golang 版本的管理与切换","number":216}}}, "staticQueryHashes": ["151096407","2861350382"]}