Boosting Large Data Transfer Speeds by Building an In-House croc Relay Server
If you are an IT professional who wastes time going through approval processes every time you move large amounts of data because security regulations prevent the use of external cloud storage, this article is the solution. croc uses the PAKE protocol, which means even the relay server cannot inspect the data. Here is how you can ensure high transfer speeds without violating security policies.
1. Eliminate Firewall Approval Waits with a Dedicated Relay Server
By placing a relay server directly in the DMZ zone instead of using external servers, you can skip the security team's approval process. croc uses TCP port 9009 for control signals and ports 9010 through 9013 for data transfer. Pinning this range makes it possible to register firewall policies.
Use Docker to run a container and isolate the environment.
`bash
docker network create --driver bridge croc-dmz-net
docker run -d --name croc-relay
-p 9009:9009 -p 9010-9013:9010-9013
-e CROC_PASS='비밀번호'
docker.io/schollz/croc:latest
`
To avoid entering the server address and password every time on internal servers, add the following configuration to your profile file (~/.bashrc).
`bash
export CROC_RELAY="내부서버IP:9009"
export CROC_PASS='비밀번호'
`
By doing this, you eliminate waiting for data transfer approvals since the data does not pass through external servers.
2. Automating the Relay Server with systemd
It is inefficient to manually start the relay every time the server reboots. Create a systemd unit file to keep the process running at all times. Using the nobody account instead of root can minimize potential damage in the event of a security incident.
Create the /etc/systemd/system/croc-relay.service file.
`ini
[Unit]
Description=croc relay service
[Service]
User=nobody
ExecStart=/usr/local/bin/croc relay --ports "9009,9010,9011,9012,9013"
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
`
After creating the file, run systemctl daemon-reload and start the service with systemctl enable --now croc-relay.service. Even if the server dies, it will automatically recover within 5 seconds.
3. Ensuring Large File Integrity and Transfer Tips
When moving project folders that are hundreds of gigabytes in size, you must prevent file loss. Use the method of archiving with tar and verifying with checksums.
Here is an example transfer script.
`bash
Sender side
tar -czf data.tar.gz ./project_folder
sha256sum data.tar.gz > checksum.txt
croc send data.tar.gz checksum.txt
Receiver side
croc recv [코드]
sha256sum -c checksum.txt && tar -xzf data.tar.gz
`
This helps avoid decompression vulnerabilities like CVE-2023-43616 and allows you to verify immediately after transfer that the data has not been corrupted.
4. Tuning Network Settings for Sustained Speed
When data transfer volume reaches the terabyte level, the default settings will cause bottlenecks. Add socket parameters to /etc/sysctl.conf to prevent sessions from dropping.
`bash
net.ipv4.tcp_keepalive_time=60
net.core.rmem_max=134217728
`
Reducing the TCP keep-alive interval to 60 seconds can solve the issue of stateful firewalls forcibly terminating sessions. If multiple teams are using this server simultaneously, allocate port ranges per team—such as 9010-9014 and 9020-9024—and run separate containers for each. Once the firewall settings are fixed, teams can perform stable transfers without interfering with one another.