Skip to main content

Unity on Unity：使用WebRTC套娃两个Unity应用

· 14 min read
Ferdinand Su

是的，我是元p

准备​

我的目标是实现一个全分布式的Unity渲染系统，显然从底层入手是不可能的，因为我对图形学或者Unity一窍不通。好在Unity引入了WebRTC协议，可以以P2P方式在计算机间传输多媒体。

先来学一下WebRTC协议吧，主要的要点是：

  • ICE：一个允许你的浏览器和对端浏览器建立连接的协议框架
  • STUN： 是一个允许位于 NAT 后的客户端找出自己的公网地址，判断出路由器阻止直连的限制方法的协议
  • NAT
  • TURN：用于穿透NAT的代理，垃圾ipv4
  • SDP：描述多媒体连接内容的协议，例如分辨率，格式，编码，加密算法等。所以在数据传输时两端都能够理解彼此的数据

有关连接的详情参阅MDN上的说明

使用C#重构的WebRTC Signalling Server​

以下是截取自Render Streaming的Server的log（public模式），ip均为内网ip：

Peer #0 is now online
Peer #1 is now online
offer: f4f9b021-e629-410f-8132-d7003de23063, sent by #1
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"offer","data":{"sdp":"v=0\r\no=- 7436445580666163562 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:sBUe\r\na=ice-pwd:EVaeRV9S2BQlCfjpA4BCQSBr\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1E:57:2D:88:62:45:FB:0E:46:5A:E3:48:E8:02:C2:EC:DB:62:90:D6:EE:C3:CD:22:1A:1E:FA:BD:11:83:1B:03\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n","datetime":1669006525165,"polite":false}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #1
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"candidate":"candidate:2743650345 1 udp 2113937151 00d8ed6a-35ba-44fa-93fd-6d343dd0da4d.local 58047 typ host generation 0 ufrag sBUe network-cost 999","sdpMLineIndex":"0","sdpMid":0,"datetime":1669006525166}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #1
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"candidate":"candidate:3306178919 1 udp 2113939711 352699e5-5008-4bae-9461-ca27c33cf18a.local 58048 typ host generation 0 ufrag sBUe network-cost 999","sdpMLineIndex":"0","sdpMid":0,"datetime":1669006525167}}
    <this candidate can't accept current signaling state Stable.>
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:2868209204 1 udp 2122260223 192.168.224.1 58051 typ host generation 0 ufrag 0KCQ network-id 6","sdpMid":"0","sdpMLineIndex":0}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:3398666143 1 udp 2122197247 2001:250:fe01:130:a428:401c:2422:c428 58052 typ host generation 0 ufrag 0KCQ network-id 2 network-cost 10","sdpMid":"0","sdpMLineIndex":0}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:3094195193 1 udp 2122131711 2001:250:fe01:130:e112:d631:4fea:46d0 58053 typ host generation 0 ufrag 0KCQ network-id 3 network-cost 10","sdpMid":"0","sdpMLineIndex":0}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:800080621 1 udp 2122063615 172.20.69.100 58054 typ host generation 0 ufrag 0KCQ network-id 1 network-cost 10","sdpMid":"0","sdpMLineIndex":0}}
answer: f4f9b021-e629-410f-8132-d7003de23063, sent by #0, to #1
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"answer","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","sdp":"v=0\r\no=- 8974781892375249057 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 58051 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 192.168.224.1\r\na=candidate:2868209204 1 udp 2122260223 192.168.224.1 58051 typ host generation 0 network-id 6\r\na=candidate:3398666143 1 udp 2122197247 2001:250:fe01:130:a428:401c:2422:c428 58052 typ host generation 0 network-id 2 network-cost 10\r\na=candidate:3094195193 1 udp 2122131711 2001:250:fe01:130:e112:d631:4fea:46d0 58053 typ host generation 0 network-id 3 network-cost 10\r\na=candidate:800080621 1 udp 2122063615 172.20.69.100 58054 typ host generation 0 network-id 1 network-cost 10\r\na=ice-ufrag:0KCQ\r\na=ice-pwd:aGzZ6c46riStENQhQlUDp3TQ\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C2:1E:96:A2:F5:ED:16:23:7A:81:97:53:99:9C:81:4A:94:C4:C3:61:65:A0:59:90:A9:0C:5A:88:5A:90:99:F7\r\na=setup:active\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n","type":"answer","polite":false}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:2868209204 1 udp 2122260223 192.168.224.1 63013 typ host generation 0 ufrag 0KCQ network-id 6","sdpMid":"1","sdpMLineIndex":1}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:3398666143 1 udp 2122197247 2001:250:fe01:130:a428:401c:2422:c428 63014 typ host generation 0 ufrag 0KCQ network-id 2 network-cost 10","sdpMid":"1","sdpMLineIndex":1}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:3094195193 1 udp 2122131711 2001:250:fe01:130:e112:d631:4fea:46d0 63015 typ host generation 0 ufrag 0KCQ network-id 3 network-cost 10","sdpMid":"1","sdpMLineIndex":1}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:800080621 1 udp 2122063615 172.20.69.100 63016 typ host generation 0 ufrag 0KCQ network-id 1 network-cost 10","sdpMid":"1","sdpMLineIndex":1}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:2868209204 1 udp 2122260223 192.168.224.1 63017 typ host generation 0 ufrag 0KCQ network-id 6","sdpMid":"2","sdpMLineIndex":2}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:3398666143 1 udp 2122197247 2001:250:fe01:130:a428:401c:2422:c428 63018 typ host generation 0 ufrag 0KCQ network-id 2 network-cost 10","sdpMid":"2","sdpMLineIndex":2}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"candidate","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","candidate":"candidate:3094195193 1 udp 2122131711 2001:250:fe01:130:e112:d631:4fea:46d0 63019 typ host generation 0 ufrag 0KCQ network-id 3 network-cost 10","sdpMid":"2","sdpMLineIndex":2}}
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
candidate: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
offer: f4f9b021-e629-410f-8132-d7003de23063, sent by #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"offer","data":{"connectionId":"f4f9b021-e629-410f-8132-d7003de23063","sdp":"v=0\r\no=- 8974781892375249057 3 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1 2 3\r\na=msid-semantic: WMS 3cac53c0-1a88-42c3-9cdc-8e9549f22c95 905f69ea-0164-4814-b6d3-6b39169ddc60\r\nm=application 58051 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 192.168.224.1\r\na=candidate:2868209204 1 udp 2122260223 192.168.224.1 58051 typ host generation 0 network-id 6\r\na=candidate:3398666143 1 udp 2122197247 2001:250:fe01:130:a428:401c:2422:c428 58052 typ host generation 0 network-id 2 network-cost 10\r\na=candidate:3094195193 1 udp 2122131711 2001:250:fe01:130:e112:d631:4fea:46d0 58053 typ host generation 0 network-id 3 network-cost 10\r\na=candidate:800080621 1 udp 2122063615 172.20.69.100 58054 typ host generation 0 network-id 1 network-cost 10\r\na=ice-ufrag:0KCQ\r\na=ice-pwd:aGzZ6c46riStENQhQlUDp3TQ\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C2:1E:96:A2:F5:ED:16:23:7A:81:97:53:99:9C:81:4A:94:C4:C3:61:65:A0:59:90:A9:0C:5A:88:5A:90:99:F7\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\nm=video 63013 UDP/TLS/RTP/SAVPF 127 119 125 118 124\r\nc=IN IP4 192.168.224.1\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate:2868209204 1 udp 2122260223 192.168.224.1 63013 typ host generation 0 network-id 6\r\na=candidate:3398666143 1 udp 2122197247 2001:250:fe01:130:a428:401c:2422:c428 63014 typ host generation 0 network-id 2 network-cost 10\r\na=candidate:3094195193 1 udp 2122131711 2001:250:fe01:130:e112:d631:4fea:46d0 63015 typ host generation 0 network-id 3 network-cost 10\r\na=candidate:800080621 1 udp 2122063615 172.20.69.100 63016 typ host generation 0 network-id 1 network-cost 10\r\na=candidate:3832978116 1 tcp 1518280447 192.168.224.1 7379 typ host tcptype passive generation 0 network-id 6\r\na=candidate:2215672687 1 tcp 1518217471 2001:250:fe01:130:a428:401c:2422:c428 7380 typ host tcptype passive generation 0 network-id 2 network-cost 10\r\na=candidate:4142814985 1 tcp 1518151935 2001:250:fe01:130:e112:d631:4fea:46d0 7381 typ host tcptype passive generation 0 network-id 3 network-cost 10\r\na=candidate:1630780957 1 tcp 1518083839 172.20.69.100 7382 typ host tcptype passive generation 0 network-id 1 network-cost 10\r\na=ice-ufrag:0KCQ\r\na=ice-pwd:aGzZ6c46riStENQhQlUDp3TQ\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C2:1E:96:A2:F5:ED:16:23:7A:81:97:53:99:9C:81:4A:94:C4:C3:61:65:A0:59:90:A9:0C:5A:88:5A:90:99:F7\r\na=setup:actpass\r\na=mid:1\r\na=extmap:1 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 urn:3gpp:video-orientation\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendonly\r\na=msid:905f69ea-0164-4814-b6d3-6b39169ddc60 b2d2ca45-fbbd-49d6-bf0b-5961987c382f\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e033\r\na=rtpmap:119 rtx/90000\r\na=fmtp:119 apt=127\r\na=rtpmap:125 red/90000\r\na=rtpmap:118 rtx/90000\r\na=fmtp:118 apt=125\r\na=rtpmap:124 ulpfec/90000\r\na=ssrc-group:FID 1172891945 1586670433\r\na=ssrc:1172891945 cname:frx+ZdAgDiN8k7hP\r\na=ssrc:1172891945 msid:905f69ea-0164-4814-b6d3-6b39169ddc60 b2d2ca45-fbbd-49d6-bf0b-5961987c382f\r\na=ssrc:1172891945 mslabel:905f69ea-0164-4814-b6d3-6b39169ddc60\r\na=ssrc:1172891945 label:b2d2ca45-fbbd-49d6-bf0b-5961987c382f\r\na=ssrc:1586670433 cname:frx+ZdAgDiN8k7hP\r\na=ssrc:1586670433 msid:905f69ea-0164-4814-b6d3-6b39169ddc60 b2d2ca45-fbbd-49d6-bf0b-5961987c382f\r\na=ssrc:1586670433 mslabel:905f69ea-0164-4814-b6d3-6b39169ddc60\r\na=ssrc:1586670433 label:b2d2ca45-fbbd-49d6-bf0b-5961987c382f\r\nm=audio 63017 UDP/TLS/RTP/SAVPF 96 97 98 102 103 104 9 0 8 99 100 101 107 108 114 106 105 13 110 112 113 126\r\nc=IN IP4 192.168.224.1\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate:2868209204 1 udp 2122260223 192.168.224.1 63017 typ host generation 0 network-id 6\r\na=candidate:3398666143 1 udp 2122197247 2001:250:fe01:130:a428:401c:2422:c428 63018 typ host generation 0 network-id 2 network-cost 10\r\na=candidate:3094195193 1 udp 2122131711 2001:250:fe01:130:e112:d631:4fea:46d0 63019 typ host generation 0 network-id 3 network-cost 10\r\na=candidate:800080621 1 udp 2122063615 172.20.69.100 63020 typ host generation 0 network-id 1 network-cost 10\r\na=candidate:3832978116 1 tcp 1518280447 192.168.224.1 7383 typ host tcptype passive generation 0 network-id 6\r\na=candidate:2215672687 1 tcp 1518217471 2001:250:fe01:130:a428:401c:2422:c428 7384 typ host tcptype passive generation 0 network-id 2 network-cost 10\r\na=candidate:4142814985 1 tcp 1518151935 2001:250:fe01:130:e112:d631:4fea:46d0 7385 typ host tcptype passive generation 0 network-id 3 network-cost 10\r\na=candidate:1630780957 1 tcp 1518083839 172.20.69.100 7386 typ host tcptype passive generation 0 network-id 1 network-cost 10\r\na=ice-ufrag:0KCQ\r\na=ice-pwd:aGzZ6c46riStENQhQlUDp3TQ\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C2:1E:96:A2:F5:ED:16:23:7A:81:97:53:99:9C:81:4A:94:C4:C3:61:65:A0:59:90:A9:0C:5A:88:5A:90:99:F7\r\na=setup:actpass\r\na=mid:2\r\na=extmap:14 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendonly\r\na=msid:3cac53c0-1a88-42c3-9cdc-8e9549f22c95 df6f8ef9-b5a6-484a-a9cc-d129fcb9bca9\r\na=rtcp-mux\r\na=rtpmap:96 opus/48000/2\r\na=rtcp-fb:96 transport-cc\r\na=fmtp:96 minptime=10;sprop-stereo=1;stereo=1;useinbandfec=1\r\na=rtpmap:97 multiopus/48000/6\r\na=fmtp:97 channel_mapping=0,4,1,2,3,5;coupled_streams=2;minptime=10;num_streams=4;useinbandfec=1\r\na=rtpmap:98 multiopus/48000/8\r\na=fmtp:98 channel_mapping=0,6,1,2,3,4,5,7;coupled_streams=3;minptime=10;num_streams=5;useinbandfec=1\r\na=rtpmap:102 ILBC/8000\r\na=rtpmap:103 ISAC/16000\r\na=rtpmap:104 ISAC/32000\r\na=rtpmap:9 G722/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:99 L16/8000\r\na=rtpmap:100 L16/16000\r\na=rtpmap:101 L16/32000\r\na=rtpmap:107 L16/8000/2\r\na=rtpmap:108 L16/16000/2\r\na=rtpmap:114 L16/32000/2\r\na=rtpmap:106 CN/32000\r\na=rtpmap:105 CN/16000\r\na=rtpmap:13 CN/8000\r\na=rtpmap:110 telephone-event/48000\r\na=rtpmap:112 telephone-event/32000\r\na=rtpmap:113 telephone-event/16000\r\na=rtpmap:126 telephone-event/8000\r\na=ssrc:967888819 cname:frx+ZdAgDiN8k7hP\r\na=ssrc:967888819 msid:3cac53c0-1a88-42c3-9cdc-8e9549f22c95 df6f8ef9-b5a6-484a-a9cc-d129fcb9bca9\r\na=ssrc:967888819 mslabel:3cac53c0-1a88-42c3-9cdc-8e9549f22c95\r\na=ssrc:967888819 label:df6f8ef9-b5a6-484a-a9cc-d129fcb9bca9\r\nm=video 63021 UDP/TLS/RTP/SAVPF 127 119 125 118 124\r\nc=IN IP4 192.168.224.1\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate:2868209204 1 udp 2122260223 192.168.224.1 63021 typ host generation 0 network-id 6\r\na=candidate:3398666143 1 udp 2122197247 2001:250:fe01:130:a428:401c:2422:c428 63022 typ host generation 0 network-id 2 network-cost 10\r\na=candidate:3094195193 1 udp 2122131711 2001:250:fe01:130:e112:d631:4fea:46d0 63023 typ host generation 0 network-id 3 network-cost 10\r\na=candidate:800080621 1 udp 2122063615 172.20.69.100 63024 typ host generation 0 network-id 1 network-cost 10\r\na=candidate:3832978116 1 tcp 1518280447 192.168.224.1 7387 typ host tcptype passive generation 0 network-id 6\r\na=candidate:2215672687 1 tcp 1518217471 2001:250:fe01:130:a428:401c:2422:c428 7388 typ host tcptype passive generation 0 network-id 2 network-cost 10\r\na=candidate:4142814985 1 tcp 1518151935 2001:250:fe01:130:e112:d631:4fea:46d0 7389 typ host tcptype passive generation 0 network-id 3 network-cost 10\r\na=candidate:1630780957 1 tcp 1518083839 172.20.69.100 7390 typ host tcptype passive generation 0 network-id 1 network-cost 10\r\na=ice-ufrag:0KCQ\r\na=ice-pwd:aGzZ6c46riStENQhQlUDp3TQ\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C2:1E:96:A2:F5:ED:16:23:7A:81:97:53:99:9C:81:4A:94:C4:C3:61:65:A0:59:90:A9:0C:5A:88:5A:90:99:F7\r\na=setup:actpass\r\na=mid:3\r\na=extmap:1 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 urn:3gpp:video-orientation\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendonly\r\na=msid:24ddeed6-4f00-4d59-82be-24aa5bf8208c 53dd1eec-f6d3-4acf-8937-78d1d0ceece3\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e033\r\na=rtpmap:119 rtx/90000\r\na=fmtp:119 apt=127\r\na=rtpmap:125 red/90000\r\na=rtpmap:118 rtx/90000\r\na=fmtp:118 apt=125\r\na=rtpmap:124 ulpfec/90000\r\na=ssrc-group:FID 727968422 1675013627\r\na=ssrc:727968422 cname:frx+ZdAgDiN8k7hP\r\na=ssrc:727968422 msid:24ddeed6-4f00-4d59-82be-24aa5bf8208c 53dd1eec-f6d3-4acf-8937-78d1d0ceece3\r\na=ssrc:727968422 mslabel:24ddeed6-4f00-4d59-82be-24aa5bf8208c\r\na=ssrc:727968422 label:53dd1eec-f6d3-4acf-8937-78d1d0ceece3\r\na=ssrc:1675013627 cname:frx+ZdAgDiN8k7hP\r\na=ssrc:1675013627 msid:24ddeed6-4f00-4d59-82be-24aa5bf8208c 53dd1eec-f6d3-4acf-8937-78d1d0ceece3\r\na=ssrc:1675013627 mslabel:24ddeed6-4f00-4d59-82be-24aa5bf8208c\r\na=ssrc:1675013627 label:53dd1eec-f6d3-4acf-8937-78d1d0ceece3\r\n","type":"offer","polite":false}}
answer: f4f9b021-e629-410f-8132-d7003de23063, sent by #1, to #0
    {"from":"f4f9b021-e629-410f-8132-d7003de23063","to":"","type":"answer","data":{"sdp":"v=0\r\no=- 7436445580666163562 3 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1 2 3\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=candidate:2743650345 1 udp 2113937151 00d8ed6a-35ba-44fa-93fd-6d343dd0da4d.local 58047 typ host generation 0 network-cost 999\r\na=candidate:3306178919 1 udp 2113939711 352699e5-5008-4bae-9461-ca27c33cf18a.local 58048 typ host generation 0 network-cost 999\r\na=ice-ufrag:sBUe\r\na=ice-pwd:EVaeRV9S2BQlCfjpA4BCQSBr\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1E:57:2D:88:62:45:FB:0E:46:5A:E3:48:E8:02:C2:EC:DB:62:90:D6:EE:C3:CD:22:1A:1E:FA:BD:11:83:1B:03\r\na=setup:passive\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\nm=video 9 UDP/TLS/RTP/SAVPF 127 119 125 118 124\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:sBUe\r\na=ice-pwd:EVaeRV9S2BQlCfjpA4BCQSBr\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1E:57:2D:88:62:45:FB:0E:46:5A:E3:48:E8:02:C2:EC:DB:62:90:D6:EE:C3:CD:22:1A:1E:FA:BD:11:83:1B:03\r\na=setup:passive\r\na=mid:1\r\na=extmap:1 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 urn:3gpp:video-orientation\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:119 rtx/90000\r\na=fmtp:119 apt=127\r\na=rtpmap:125 red/90000\r\na=rtpmap:118 rtx/90000\r\na=fmtp:118 apt=125\r\na=rtpmap:124 ulpfec/90000\r\nm=audio 9 UDP/TLS/RTP/SAVPF 96 103 104 9 0 8 106 105 13 110 112 113 126\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:sBUe\r\na=ice-pwd:EVaeRV9S2BQlCfjpA4BCQSBr\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1E:57:2D:88:62:45:FB:0E:46:5A:E3:48:E8:02:C2:EC:DB:62:90:D6:EE:C3:CD:22:1A:1E:FA:BD:11:83:1B:03\r\na=setup:passive\r\na=mid:2\r\na=extmap:14 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:96 opus/48000/2\r\na=rtcp-fb:96 transport-cc\r\na=fmtp:96 minptime=10;useinbandfec=1\r\na=rtpmap:103 ISAC/16000\r\na=rtpmap:104 ISAC/32000\r\na=rtpmap:9 G722/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:106 CN/32000\r\na=rtpmap:105 CN/16000\r\na=rtpmap:13 CN/8000\r\na=rtpmap:110 telephone-event/48000\r\na=rtpmap:112 telephone-event/32000\r\na=rtpmap:113 telephone-event/16000\r\na=rtpmap:126 telephone-event/8000\r\nm=video 9 UDP/TLS/RTP/SAVPF 127 119 125 118 124\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:sBUe\r\na=ice-pwd:EVaeRV9S2BQlCfjpA4BCQSBr\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1E:57:2D:88:62:45:FB:0E:46:5A:E3:48:E8:02:C2:EC:DB:62:90:D6:EE:C3:CD:22:1A:1E:FA:BD:11:83:1B:03\r\na=setup:passive\r\na=mid:3\r\na=extmap:1 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 urn:3gpp:video-orientation\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:119 rtx/90000\r\na=fmtp:119 apt=127\r\na=rtpmap:125 red/90000\r\na=rtpmap:118 rtx/90000\r\na=fmtp:118 apt=125\r\na=rtpmap:124 ulpfec/90000\r\n","datetime":1669006525605}}
Peer #1 is now offline
Peer #2 is now online
connect: 4b847179-bc61-44da-984c-3196c7fa4dd2 set up, suggested by #2
offer: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #2
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #2
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #2
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
answer: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0, to #2
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
offer: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
answer: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #2, to #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 4b847179-bc61-44da-984c-3196c7fa4dd2, sent by #0
candidate: 657da968-2f35-47d8-97bd-92c9ae0dd22c, sent by #0
candidate: 657da968-2f35-47d8-97bd-92c9ae0dd22c, sent by #0
candidate: 657da968-2f35-47d8-97bd-92c9ae0dd22c, sent by #0
candidate: 657da968-2f35-47d8-97bd-92c9ae0dd22c, sent by #0
candidate: 657da968-2f35-47d8-97bd-92c9ae0dd22c, sent by #0
candidate: 657da968-2f35-47d8-97bd-92c9ae0dd22c, sent by #0
candidate: 657da968-2f35-47d8-97bd-92c9ae0dd22c, sent by #0
candidate: 657da968-2f35-47d8-97bd-92c9ae0dd22c, sent by #0
Peer #3 is now offline

翻译过来的流程大概是（S：服务端；C：客户端；G：信号服务器）：

S连接G；
C连接G；
C向G注册连接；
C向G发送offer，G向C外的peers广播；
C向G发送candidate(若干次)，G向C外的peers广播；
S向G发送candidate(若干次)，G向S外的peers广播；
S经由G发送answer给C；
S向G发送candidate(若干次)，G向S外的peers广播；
S向G发送offer，G向S外的peers广播；
C经由G发送answer给S；
S向G发送candidate(若干次)，G向S外的peers广播；

重新实现的C#信号服务器应该具有相同的逻辑。经过一周的调试后，它终于能跑了：

甚至能跑

Unity Over Unity：在Unity中接受另一个Unity发来的视频流​

Unity Render Streaming已经提供了Receiver功能的一个Example，因此先我们不改动它的接收端；如果发送端采取Broadcast，则可以直接使用；否则，发送端可以使用SingleConnetion替换掉原来的Broadcast，并把信号服务器改为Private模式，然后做出以下几项更改(如图)：

甚至能跑

  1. (图里的2)添加一个Single Connection的Component，把之前Broadcast里的stream复制进去，再删掉原有的Broadcast
  2. (图里的1)把RenderStreaming里的Handlers改成新加入的Single Connection
  3. 关掉硬件加速，因为客户端不支持H. 264硬编码的版本(?)，文档里说关掉就好了
  4. 加入一个新的Script，暂且叫Single Connection Driver吧，代码如下：
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using Unity.RenderStreaming;
using UnityEngine;
using UnityEngine.UI;
using Unity.RenderStreaming.Samples;
using Unity.RenderStreaming.Signaling;
using Random = UnityEngine.Random;

public class SingleConnectionDriver : MonoBehaviour
{
#pragma warning disable 0649
    [SerializeField] private RenderStreaming renderStreaming;
    [SerializeField] private SingleConnection singleConnection;
#pragma warning restore 0649

    private string connectionId="crossport.pt1";
    
    void Start()
    {
        if (!renderStreaming.runOnAwake)
            renderStreaming.Run(
            hardwareEncoder: false,
            handlers: new SignalingHandlerBase[]{singleConnection});
        singleConnection.CreateConnection(connectionId);
    }
    private void OnDestroy()
    {
        singleConnection.DeleteConnection(connectionId);
    }

}

然后以此启动Crossport，服务端和客户端即可得到以下效果：

甚至能跑

重定向加入和Crossport注册​

因为UoU整个系统会是一个大型的分布式系统，同时支持多套应用、多个组件，必然会涉及服务发现/注册和广播域分割的问题。

服务发现/注册的实现有许多，但是显然和Unity都无关，因此必须在Crossport上建立服务发现/注册的机制；广播域分割显然也是信号服务器的任务。

而在Asp.NET上，最好的服务发现/注册机制的方式很多，我们可以暂缓考虑。但是无论如何都会涉及到一个问题：客户端接入到当前的信号服务器，而所请求的服务未必连接在当前服务器上，因此必然会出现一个请求转发问题。一般来说有以下几种方案：

  • DNS或者底层路由的重定向
  • HTTP 30X的重定向
  • 反向代理

DNS或者底层路由的重定向显然不够灵活，而反向代理占用资源很多，因此我选择了HTTP 30X的重定向作为请求转发方案。

重定向​

WebSocketSharp支持连接阶段的Http重定向(StatusCode 30X)，但是默认关闭。因此，需要自己重写一套WebsocketSignalling，其中只修改了WSCreate段(加入一行m_webSocket.EnableRedirection = true;)：

private void WSCreate()
{
  m_webSocket = new WebSocket(m_url);
  if (m_url.StartsWith("wss"))
  {
      m_webSocket.SslConfiguration.EnabledSslProtocols =
          SslProtocols.Tls | SslProtocols.Tls11 | SslProtocols.Tls12;
  }
  m_webSocket.EnableRedirection = true;
  m_webSocket.OnOpen += WSConnected;
  m_webSocket.OnMessage += WSProcessMessage;
  m_webSocket.OnError += WSError;
  m_webSocket.OnClose += WSClosed;
  Monitor.Enter(m_webSocket);
  Debug.Log($"Signaling: Connecting WS {m_url}");
  m_webSocket.ConnectAsync();
}

因为不再使用RenderStreaming包自带的Signalling，RenderStreaming不再支持RunOnAwake，需要我们手写一个Driver来进行驱动：

public class CrossportDriver : MonoBehaviour
{
#pragma warning disable 0649
    [SerializeField] private RenderStreaming renderStreaming;
    [SerializeField] private string url;
    [SerializeField] private CrossportConfig config;
    [SerializeField, Tooltip("Time interval for polling from signaling server.")]
    private float interval = 5.0f;
#pragma warning restore 0649
    public void Awake()
    {
        if(renderStreaming.runOnAwake)
            renderStreaming.Stop();
        renderStreaming.Run(signaling: new CrossportSignalling(url, interval, config, SynchronizationContext.Current));
    }
}

这里的CrossportConfig是用于广播域的配置，暂时是收费节目，先不管它。CrossportDriver是需要加入到RenderStreaming GameObject的Component，然后四个SerializeField均需要在Editor里手动配置参数。

然后，服务端便可以return Redirect("新地址");来完成连接的转发，而不需要架设代理了。

甚至能跑