1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
|
What is Shard?
===============
**WARNING: This is a primer on what I want Shard to become. As of today I'm
only starting work on this project and there is almost nothing to see. I hope
this will evolve and approach at least a significant portion of the goals
identified below.**
Shard is a decentralized communication system adapted to public and private
discussion and collaboration.
The system is based on distributed objects called "shards" of different types
that users share and collaborate on.
Shards are made to be usable offline or on small networks disconnected from the
broader Internet, with updates becoming synchronized automatically whenever
peers are able to communicate.
Shard compared to other systems
===============================
Shard is born from the synthesis of ideas already found in other networks, but
aiming to fix some of their shortcomings.
Shard vs. ZeroNet
-----------------
Just like ZeroNet, Shard will enable users to run public applications with
distributed collaboration. We wish to improve on ZeroNet in the following
significant manners:
- Full support for one-to-one and groupwise private communication and
collaboration. Examples of such in ZeroNet are quite limited, for example
ZeroMail enables user to exchange private emails but requires all peers
participating in the ZeroMail system to store all messages sent by
everyone. We want to make this work better by only requiring participants
to store the private information.
- Collaboration on very large databases. We wish to be able to store big
collaborative databases (e.g. wikipedia) in a distributed way, so that
it can be run in a decentralized fashion without requiring everyone to store
a full copy.
- Performance improvements. This is done by using a language that is by
design well adapted to network programing with many things going on at once
(Elixir/Erlang), as well as by optimizing our data stores to enable fast
querying.
Shard vs. Secure Scuttlebutt
----------------------------
Just like SSB, Shard will enable users to work off grid and exchange data when
a network connection happens to be available. We wish to improve upon the
following aspects of SSB:
- More granularity on what peers store or not, by splitting a user's
contribution in different "shards" that correspond to different topics.
Users can still chose to replicate the shards that their friends use so
that they have more options to synchronize and backup their data, but
this is not required as it is in SSB where you have to replicate all the
data of your friends and their friends.
- Decentralized discovery: we don't want to rely on centralized servers such
as SSB pubs. We want to use DHT technology as a primary means of discovering
new people and finding contents.
- We wish to be very general from the start, and not limited to social-media-like
conversations.
- We want excellent support for private one-to-one and groupwise communications.
Shard vs. Matrix
----------------
We aim to be much more general than a chat room, supporting collaborative
edition, file sharing, any kinds of databases, etc.
We also aim to work in a completely distributed fashion and not require
federation servers.
Shard vs. Mastodon / GNU Social
-------------------------------
Same remarks than for Matrix.
Current status
==============
What is available
-----------------
All of these are rudimentary prototypes at an early stage.
* Chat rooms (public and private) with full history and efficient data structure for
retrieving missing messages after disconnection
* File upload (public only)
* Directories as collections of links to shards
What is missing
---------------
See `TODO` file for more details.
* Finding peers via DHT (very easy to add)
* Invite/notification system
* Good access control
* Good networking behind NAT/Firewall
* Automatic discovery on local networks
* More applications
How to use it?
==============
Installation
------------
Shard has been tested under Linux and should probably work also under Mac OS.
No idea about MS Windows.
We don't have an automated installation procedure for now. Sorry.
First, install the git version control system, as well as the Erlang and Elixir
programming languages. This should be done using your favourite package
manager.
If you wish to use the web UI, you must also install nodejs and npm.
Then, clone the Shard repository using git:
```
git clone git://adnab.me/shard.git
```
### Web UI version
Enter the shard directory and create a data2 folder.
```
cd shard
mkdir data2 # for web UI
```
Enter the shardweb subdirectory and install Elixir dependencies.
```
cd shardweb
mix deps.get
mix compile
```
Build the web assets:
```
cd assets
npm install
cd ..
```
**The default configuration assumes you have cloned the shard repo in your
`$HOME`. If this is not the case, ajust the data path parameter in
`shardweb/config/config.exs` accordingly.**
### CLI version only
Enter the shard directory and create a data folder.
```
cd shard
mkdir data # for CLI use only
```
Enter the shard subdirectory and install Elixir dependencies.
```
cd shard
mix deps.get
mix compile
```
**The default configuration assumes you have cloned the shard repo in your
`$HOME`. If this is not the case, ajust the data path parameter in
`shard/config/config.exs` accordingly.**
Web Ui usage
------------
Once you have completed the installation steps above, Shard web can be started
by running the following in the `shardweb` directory of the repo:
```
iex -S mix phx.server
```
The UI will become accessible at address `http://127.0.0.1:4000`.
CLI usage
---------
Once you have completed the installation steps above, Shard can be started by
running the following in the `shard` directory of the repo:
```
iex -S mix
```
Once Shard has started and you have an iex prompt, you can enter the minimalist
CLI interface by typing:
```
SCLI.run
```
This CLI supports a few basic commands:
- `/connect 1.2.3.4 4044`: connect to a Shard peer at IP 1.2.3.4 on port 4044 (default port)
- `/join test`: join chatroom called test
- `/nick my_nickname`: set nickname
- `/list`: list channels we are currently connected to
- `/hist`: show recent messages on current channel
- `/pm nickname1 [nickname2] [...]`: enter private conversation with someone
- `/send_file path`: make file available on the network and send link to current chat room.
**WARNING: all files are publicly available for now, even if they are sent in a private chat room.**
- `/shards`: return the list of all shards on the system
- `/info [file_shard_uri]`: shows progress of a file download. Starts download if not already started.
- `/quit`: return to iex prompt
|
