Documentation

If you like code, there is Doxygen generated documentation (HTML) available that you may want to look at. The HTML converted man pages are now also available.

Table of Contents

1. Copyright and Contributions
2. Design Goals and Philosophy
3. Installation
4. User Interface
5. Concepts
6. The GNUnet configuration file
7. System Design
8. GNUnet Protocol I: node-to-node
9. GNUnet Protocol II: client-to-node
10. Code Modules
11. Future Work

Some fairly special topics that are mostly relevant for very advanced users and developers have their own dedicated page:

1. Transport services
2. Content encoding
3. Namespaces and directories

Copyright and Contributions

Design Goals and Philosophy

GNUnet's primary design goals are to protect the privacy of its users and to guard itself against attacks or abuse. GNUnet does not have any mechanisms to control, track or censor users. Instead, the GNUnet protocols aim to make it as hard as possible to find out what is happening on the network or to disrupt operations.

We call GNUnet a network because we want to support any operation that is typically performed on a network. While the first versions only support anonymous file-sharing, other applications such as mail, news or distributed computation will hopefully follow in the future. Running additional applications over the link-to-link encrypted peer-to-peer infrastructure increases the anonymity of the peers as the set of users and the amount of encrypted traffic grows. Increasing privacy is the primary reason why GNUnet is developed to become a peer-to-peer framework. A rudimentary chat client has been implemented as reference code for how to implement other applications on top of the GNUnet peer-to-peer infrastructure.

For GNUnet, efficiency is not paramount. If there is a more secure and still practical approach, we would choose to take the more secure alternative. telnet is more efficient than ssh, yet it is obsolete. Hardware gets faster, and code can be optimized. Fixing security issues as an afterthought is much harder.

While security is paramount, practicability is still a requirement. The most secure system is always the one that nobody can use. Since individual security requirements may vary, the only good solution here is to allow individuals to trade-off security and efficiency. The primary challenge in allowing this is to ensure that the economic incentives work properly. In particular, this means that it must be impossible for a user to gain security at the expense of other users. Many designs (e.g. anonymity via broadcast, see P5) fail to give users an incentive to choose a less secure but more efficient mode of operation. GNUnet should avoid whereever possible to rely on protocols that will only work if the participants are benevolent. While some designs have had widespread success while relying on parties to observe a protocol that may be sub-optimal for the individuals (e.g. TCP Nagle), a protocol that ensures that individual goals never conflict with the goals of the group is always preferable.

Anonymity can always only be achieved with large numbers of participants. Thus it is important that we make GNUnet easy to use and well-documented.

Installation

# ./configure --prefix=$HOME
# make
# make install

# ./configure --prefix=$HOME --with-extractor=$HOME
# make
# make install

# mkdir $HOME/.gnunet
# cp contrib/gnunet.conf $HOME/.gnunet/gnunet.conf

Finally, start the server using

# gnunetd

User Interface

gnunet-insert

# gnunet-insert [-n] -f FILENAME "Description" KEYWORDS*

By default, GNUnet indexes a file instead of copying it. This is much more efficient, but requries the file to stay unaltered at the location where it was when it was indexed. If you intend to move, delete or alter a file, consider using the option -n which will force GNUnet to make a copy of the file in the database. Since it is much less efficient, this is strongly discouraged for large files. When GNUnet indexes a file (default), GNUnet does not create an additional encrypted copy of the file but just computes a summary (or index) of the file. That summary is approximately two percent of the size of the original file and is stored in GNUnet's database. Whenever a request for a part of an indexed file reaches GNUnet, this part is encrypted on-demand and send out. There is no need for an additional encrypted copy of the file to stay anywhere on the drive. This is very different from other systems, such as Freenet where each file that is put online must be in Freenet's database in encrypted format, doubling the space requirements if the user wants to preseve a directly accessible copy in plaintext.
Thus indexing should be used for all files where the user will keep using this file (at the location given to gnunet-insert) and does not want to retrieve it back from GNUnet each time.
The option -n may be used if the user fears that the file might be found on his drive (assuming the computer comes under the control of an adversary). When used with the -n flag, the user has a much better chance of denying knowledge of the existence of the file, even if it is still (encrypted) on the drive and the adversary is able to crack the encryption (e.g. by guessing the keyword).
Multiple files can be inserted in one run with the gnunet-insert-multi command. If you want to remove a file that you have indexed from the local peer, use the tool gnunet-delete to un-index the file.

Before you can insert, index, share, search or download files from GNUnet you must start the GNUnet server, gnunetd. gnunetd has the following common options:

# gnunetd -c CONFIGFILE -u USER 

The command gnunet-search can be used to search for content on GNUnet. The format is

# gnunet-search [-t TIMEOUT] KEYWORD [AND KEYWORD]*

# gnunet-search Das Kapital

# gnunet-search "Das Kapital"

Search results are printed by gnunet-search like this:

gnunet-download -o "COPYING" -- 77DBE6F971D66F641B2262DDCE78CA8FB6815E60\
  E34BFD7843D4C8469EFC3DAD260F1F71783E2A87 466DC92 17992
=> The GNU Public License <= (mimetype: unknown)

In order to download a file, you need the three values returned by gnunet-search. You can then use the tool gnunet-download to obtain the file:

# gnunet-download -f FILENAME -- HASH1 HASH2 CRC SIZE

# gnunet-download -o "COPYING" -- 77DBE6F971D66F641B2262DDCE78CA8FB6815E60\
  E34BFD7843D4C8469EFC3DAD260F1F71783E2A87 466DC92 17992

The option -c CONFIGFILE can be passed to each of the commands to override the default location of the configuration file. The option -v shows the current version number. Use -h to get a short description of the options.

gnunet-stats

# bin/gnunet-stats
Uptime (seconds)                                            :              115
# kb of storage used                                        :           402821
# kb of storage remaining                                   :           645755
% of allowed network load                                   :               42
% of allowed cpu load                                       :                0
# bytes of noise received                                   :              200
# bytes received from clients                               :               16
# times outgoing msg sent (bandwidth ok)                    :                6
# times outgoing msg dropped (bandwidth stressed)           :                0
# times incoming msg accepted (cpu ok)                      :               10
# times incoming msg dropped (cpu overloaded)               :                0
# ping messages sent                                        :                1
# ping messages received                                    :                2
# pong messages sent                                        :                2
# pong messages received                                    :                1
# sessionkeys received                                      :                1
# valid sessionkeys received                                :                0
# sessionkeys sent                                          :                1
# connections shutdown                                      :                0
# currently connected nodes                                 :                1
# bytes noise sent                                          :             2656
# encrypted bytes sent                                      :             7260
# bytes decrypted                                           :            10164
# bytes received via tcp                                    :            22744
# bytes sent via tcp                                        :             9076
# bytes received via udp                                    :            15794
# bytes sent via udp                                        :             8016
# HELO messages received from http server                   :               29
# HELO messages received overall                            :                7
# valid HELO messages received                              :                2
# HELO messages forwarded from other peers                  :                0
# HELO messages originated                                  :                0
# Bloomfilter (content_bloomfilter) hits                    :                0
# Bloomfilter (content_bloomfilter) misses                  :               34
# Bloomfilter (content_bloomfilter) adds                    :                0
# Bloomfilter (content_bloomfilter) dels                    :                0
# Bloomfilter (keyword_bloomfilter) hits                    :                0
# Bloomfilter (keyword_bloomfilter) misses                  :              196
# indexed files                                             :                4
# size of indexed files                                     :          1544183
# lookup (3HASH, search results)                            :                0
# lookup (CHK, inserted or migrated content)                :                1
# lookup (ONDEMAND, indexed content)                        :                0
# times storage full                                        :                0
# kb content pushed out as padding                          :                0
# kb downloaded by clients                                  :                0
# kb ok content in                                          :                0
# kb dupe content in                                        :                0
# kb orphan or pushed content in                            :                1
# p2p queries received                                      :               45
# p2p CHK content received (kb)                             :                1
# p2p search results received (kb)                          :                0
# client queries received                                   :                0
# client CHK content inserted (kb)                          :                0
# client 3HASH search results inserted (kb)                 :                0
# client file index requests received                       :                0
# file index requests received                              :                0
# super query index requests received                       :                0

gnunet-gtk

After gnunet-gtk was started, you should see the following:

In order to insert a file, use the File menu. A dialog will pop up and prompt you to choose the file to insert:

Select the file and press ok. Next you will have to provide a description and keywords and choose if you want to index or insert the file (choose index if the file will not be moved or deleted to safe space):

Finally, press ok and watch GNUnet making the file available:

You can even do these things in parallel:

Concepts

Anonymity

Contrary to other designs, we do not believe that users achieve anonymity just because their requests are obfuscated by a couple of indirections. This is not sufficient if the adversary uses traffic analysis. In our model, the adversary is very powerful. We assume that the adversary can see all the traffic on the Internet. And while we assume that the adversary can not break our encryption, we assume that the adversary has many participating nodes in the network and that it can thus see many of the node-to-node interactions since it controls some of the nodes.
GNUnet is build on the idea that users can be anonymous if they can hide their actions in the traffic created by other users. Hiding actions in the traffic of other users requires participating in the traffic, bringing back the traditional technique of using indirection and source rewriting. Source rewriting is required to gain anonymity since otherwise an adversary could tell if a message originated from a host by looking at the source address. If all packets look like they originate from a node, the adversary can not tell which ones originate from that node and which ones were routed. Note that in this midset, any node can decide to break the source-rewriting paradigm without violating the protocol, as this only reduces the amount of traffic that a node can hide its own traffic in.

If we want to hide our actions in the traffic of other nodes, we must make our traffic indistinguishable from the traffic that we route for others. As our queries must have us as the receiver of the reply (otherwise they would be useless), we must put ourselves as the receiver of replies that actually go to other hosts; in other words, we must indirect replies. Unlike other systems, with GNUnet we do not have to indirect the replies if we don't think we need more traffic to hide our own actions. This increases the efficiency of the network as we can indirect less under higher load.

Deniability

Authentication

Cryptography

Accounting

Confidentiality

The GNUnet configuration file

The configuration of the SMTP transport layer is described here.

NETWORK: HOST

NETWORK: PORT

NETWORK: INTERFACE

NETWORK: IP

NETWORK: HELOEXCHANGE

NETWORK: TRUSTED

LOAD: INTERFACES

LOAD: BASICLIMITING

LOAD: MAXNETBPSUPTOTAL

LOAD: MAXNETBPSDOWNTOTAL

LOAD: MAXCPULOAD

UDP: PORT

UDP: BLACKLIST

UDP: MTU

TCP: PORT

TCP: BLACKLIST

TCP: MTU

TCP: MAXTCPCONNECTS

AFS: DISKQUOTA

AFS: ANONYMITY-RECEIVE

• If the value v is smaller than 1000, it means that if GNUnet routes n bytes of messages from foreign peers, it may originate n/v bytes of queries in the same time-period. The time-period is twice the average delay that GNUnet deferrs forwarded queries.
• If the value v is larger or equal to 1000, it means that if GNUnet routes n bytes of QUERIES from at least (v % 1000) peers, it may originate n/v/1000 bytes of queries in the same time-period.

AFS: ANONYMITY-SEND

AFS: ACTIVEMIGRATION

AFS: INDIRECTIONTABLESIZE

AFS: SEARCHTIMEOUT

AFS: EXTRACTORS

AFS: DATABASETYPE

GNUNETD: HELOEXPIRES

GNUNETD: MAXCONNECT

GNUNETD: LOGLEVEL

GNUNETD: LOGFILE

GNUNETD: TIMESTAMPS

GNUNETD: PIDFILE

GNUNETD: HOSTS

# cat ~/.gnunet/data/hosts/* > hostlist

Once connected, GNUnet hosts exchange information about other hosts automatically. Thus except for the initial connection, there should be no pressing need to obtain a new list (except if a node was offline for a long time and the old list aged so much that it became useless). If hosts cannot be reached and the time that the key has been signed to be valid by the sender has expired, GNUnet deletes their identities from data/hosts/. Note that the trust information is kept "forever".

GNUNETD: HOSTLISTURL

GNUNETD: APPLICATIONS

GNUNETD: TRANSPORTS

GNUNET-INSERT: CONTENT-PRIORITY

The contrib/ directory contains a template that should be self-explanatory.

System Design

The util/ directory contains utility methods that could even be useful without GNUnet (io, cron, semaphores, etc.). util/ is thus used throughout the system. util/ contains also cryptographic primitives and the configuration system.
The test/ directory contains (always too few) testcases. The historical/ directory contains old code that is not used anymore but maybe useful in the future. The include/ directory contains all the include files in a directory structure that matches the tree for the code.

In GNUnet, the server gnunetd is responsible for accounting, routing and link-to-link encryption. The core relies on implementations of the TransportAPI for the actual transport of packets. The transport layer has, like the internet protocol (IP), best-effort semantics. There is no guarantee that a message will be delivered.
Services (like anonymous file-sharing) are build on top of GNUnet. Services use the CoreAPIForApplication to access the GNUnet core. Services are responsible for adding reliability (through retransmission) to the networking layer. The service is also responsible for avoiding congestion (see TCP). While the core will enforce bandwidth limitations set by the user, services should implement better strategies.
In order to ease the implementation of a service, GNUnet splits services in two parts. The first part resides in the process space of gnunetd and is always available to the GNUnet core. The core will notify this lowest layer of the service of messages that have arrived for the service. This layer of the service is responsible for all node-to-node interactions (for AFS, this layer provides routing, content lookup and handles QUERY/CONTENT messages). The second part of every service is an application which is invoked by the user. The idea is that the user starts a sepearate program (like gnunet-search, gnunet-download or gnunet-gtk) which provides a user interface to the service. This user interface uses a trusted (local) TCP connection to talk to the in-process part of the service.
The GNUnet core provides an easy way for services to register for certain types of messages from the application. The tcpio implementation provides functions to send and receive messages via TCP. Note that the TCP connection between the application and gnunetd is presumed to be totally secure (i.e. via loopback). You can specify the list of trusted IP addresses (i.e. the LAN) that are allowed to connect as clients to GNUnetd.

We now describe how you can write your own applications, interfaces and transport mechanisms for the GNUnet peer-to-peer framework.

How do I write a new application service for GNUnet?

1. The first is a dynamic library that plugs into the GNUnet core. This library must provide a single function which is invoked by the core when gnunetd starts. This function registers a couple of callbacks with the GNUnet core in order to handle certain peer-to-peer messages. Every peer-to-peer message is bounded in size (1300 octets) and must be bound to the appropriate application module using a number typically defined in src/include/util/ports.h. The number must be globally unique. The core module will be called whenever a peer-to-peer message matching a registered port number is received. The core module can send messages to other nodes using the CoreAPIForApplication. Typically, the module will also make use of GNUnet's client-server implementation tcpserver to communicate with a user interface. Like with peer-to-peer messages, the service can register for client-server messages.
2. The second piece of code is a user interface application. Some implementations may even use a set of user interface applications (for example, the anonymous file sharing has gnunet-search, gnunet-insert, gnunet-download and a GUI that provides the combined functionality, gnunet-gtk). It depends on the specifics of the application if it uses dynamic libraries or not. The application must communicate with the service module that is loaded into the GNUnet core. While any IPC mechanism should theoretically work, the preferred way that is used by all existing GNUnet applications is tcpio, a simple TCP connection that communicates with the tcpserver.

   The simplest way for a user interface to connect to the GNUnet core via TCP is to use the helper methods defined in the common library, which defines methods for parsing the command line options and getting a client socket that is connected to gnunetd.

How do I write a new transport mechanism for GNUnet?

Every new transport mechanism must define an identification number in ports. Note that this number is used purely internally in GNUnet and does not have to correspond to the underlying protocol in any way (though of course it can help users to understand which transports 25, 17 or 6 are if these numbers magically match well-known ports or IP protocol numbers).

For details on the transport API see our transport paper.

How do I write a new user interface for GNUnet?

The next step is to examine the small shell-tools to get an idea how the application specific libraries work and to evolve the shell-tools into a user interface. You may also decide to just invoke the shell-tools as separate processes that do the real work, though this is much less efficient and less powerful (thus this is not the recommended approach).

Threading and Synchronization

1. The core may call registered callback handlers at any time, and also concurrently. The application modules are responsible for synchronizing access to their internal state properly. In practice, client code will use MUTEX_CREATE in initializers and then guard access to mutable global shared state using MUTEX_LOCK and MUTEX_UNLOCK.
2. In order to avoid deadlocks, code that was called via callback from the core may not invoke methods on the core while holding locks of the application. The rationale behind this rule is that the core may have a thread A that holds an internal lock L and calls a callback on the service. If simultaneously another thread B holds a lock S of the service and calls back on the core, this may result in a deadlock if that callback blocks trying to aquire lock L and thread A blocks trying to aquire lock S. If thread A does not hold any internal locks when calling on the core, this situation can be safely avoided.

   Programming with this paradigm is not very difficult. Still, it often requires a simple trick if the callback to the core requires an argument that is typically kept in the shared global state. The trick is, to copy the shared global state into a local buffer while holding the lock, then releasing the lock and finally doing the callback on the core. An example for this behavior can be found in the querymanager code.

3. Core threads should never be blocked indefinitely. The core has a limited number of processing threads. If clients (service modules in the core or applications connecting via TCP) block these threads, the core will stop working. Disk-IO is typically not problematic (though excessive synchronous random IO operations may degrade performance significantly), and synchronization with other threads is also no problem (as long as they do not deadlock). Encryption operations are also not the problem. The only problem in practice are blocking socket operations. Blocking socket operations occur pretty much only apply to TCP sockets.

   The TCP transport layer avoids blocking operations using select calls and IO buffers. If a message can not be buffered (buffer full) and sending would block, the TCP transport layer discards the message (or: why we like unreliable operational semantics). Client-server TCP connections are more problematic. First of all, the client-server connection has reliable semantics. Thus the gnunetd tcpio code blocks on writes to the client. Similarly, the client blocks when sending messages to gnunetd. Since the client-server TCP connection is supposed to use a fast (loopback, LAN) connection and since gnunetd uses a thread per client connection, blocking on this connection is acceptable. The problem with blocking here is that it leaves the possibility of an inter-process deadlock.

   The deadlock can occur if the TCP thread that receives and processes messages from the client blocks on a write to the client (TCP buffer queue full) and thus gnunetd no longer reads from the TCP pipe. If the client blocks on a write to gnunetd and thus no longer reads from its end of the TCP connection, both processes block forever. This problem is typically hard to diagnose since it involves two processes and may only occur after the TCP buffers of the operating system are full. A typical symptom is that netstat -tn shows two local TCP connections with very full receive and send buffers that do not change.

   The solution to the problem is that every client (not peer, remember that clients are trusted and can thus be expected to follow the protocol correctly) must always be in a state where it has a thread that can receive and process messages from gnunetd. That thread should never block, neither by doing a direct write to gnunetd nor by aquireing a lock that could be hold by another thread while that thread writes to gnunetd. A typical solution to this problem is to use one thread that processes replies and another thread that generates requests. The RequestManager code is an example for this code. It uses the cron thread to send requests to gnunetd and a separate processing thread to process replies.

GNUnet Protocol I: node-to-node

• HELO
• SKEY
• PING
• PONG
• TIMESTAMP
• SEQUENCE
• NOISE
• HANGUP
• QUERY
• 3HASH
• CHK

The following diagram illustrates a possible sequence of messages:

Nodes then exchange service messages like the QUERY, CHK and 3HASH messages. TIMESTAMP and SEQUENCE numbers can be used to prevent a malicious adversary to replay packets. NOISE must be used to make packets look uniform in size. The MTU is determined by the transport layer and advertised in the HELO message. HANGUP can be used by any of the nodes to drop the connection. A connection that is inactive for a long time (about 15 minutes) is also considered dropped. The HANGUP is neither acknowledged nor required.

A packet exchanged between GNUnet hosts can contain any number of messages (only limited by the MTU of the transport layer).

The transport layer implementation is responsible for encapsulating the message appropriately. In addition to the message itself, the transport mechanism must communicate:

• The sender identity (hash of public key)
• If the message is in plaintext or encrypted
• The CRC of the (plaintext) message
• The size of the message

The current UDP implementation encapsulates messages in this format:

The size of the UDP packets is by default 1472 octets, the advertised MTU to the GNUnet core is 1444 octets. If your network MTU is smaller than 1500 octets (ethernet), you may want to specify a different MTU in the configuration.

The encryption (blowfish) of the noise-padded body is done by the GNUnet core, the transport implementation only needs to transmit the information listed above. For received packets, the transport is not responsible for checking the CRC. The GNUnet core will decrypt the message (if applicable) and compute the CRC over the body of the packet (all messages) in plaintext. If the CRC check fails the core will discard the message silently.

Each of the messages in the body has the form

HELO

Receivers of HELOs must verify the signature and check that the HELO has not expired. Nodes should delete HELOs that have expired for a long time. HELOs that expired just a short time ago may belong to nodes that may re-appear. GNUnet nodes may still keep trying to connect to these nodes, but their HELOs should no longer be propagated. HELOs can be send to hosts without an established encrypted connection (in plaintext). This is necessary because HELOs and SKEYs are needed to form the encrypted channel in the first place. Once an encrypted channel was formed, nodes can exchange HELOs via that channel.

For the UDP and TCP transport mechanisms, the sender address has this format:

SKEY

The sender of a session key not only encrypts the key with the public key of the receiver but also signs it (together with an creation time) with its own private key. The sender must remember the sessionkey and can start using it after receiving an acknowledgement (in the form of a PING) from the receiver.
The format of an SKEY message is the following:

PING

PONG

TIMESTAMP

SEQUENCE

NOISE

HANGUP

The anonymous file-sharing service (AFS) defines three messages:

QUERY

Multi-queries do not exist for queries that search for content.

3HASH

CHK

Additional applications may define more messages. The assigned numbers should be noted in ports.h. Experimental messages should use high numbers (> 60000).

GNUnet Protocol II: client-to-node

• CS_RETURN_VALUE
• CS_GET_STATISTICS
• CS_STATISTICS
• CS_QUERY
• CS_3HASH
• CS_CHK
• CS_INSERT_CHK
• CS_INSERT_3HASH
• CS_INDEX_BLOCK
• CS_INDEX_FILE
• CS_INDEX_SUPER

The only generic client-server (CS) message that are currently defined are RETURN_VALUE and GET_STATISTICS and STATISTICS. All other CS messages are specific to anonymous file-sharing (AFS).

CS_RETURN_VALUE

CS_GET_STATISTICS

CS_STATISTICS

CS_QUERY

CS_3HASH

CS_CHK

CS_INSERT_CHK

CS_INSERT_3HASH

CS_INDEX_BLOCK

CS_INDEX_FILE

CS_INDEX_SUPER

Code Modules

• util/configuration
• server/handler
• server/tcpserver
• server/heloexchange
• server/trustservice
• server/connection
• server/keyservice
• server/pingpong
• server/knownhosts
• afs/database/
• afs/protocol/routing

util/configuration

server/handler

server/tcpserver

server/heloexchange

server/trustservice

server/connection

server/keyservice

server/pingpong

server/knownhosts

afs/database/

afs/protocol/routing

The routing modul is implemented as a large hashtable. In the case of collisions, the entry with the smallest remaining time-to-live is discarded. Time-to-lives in queries are bounded by the query priority to prevent malicious peers from taking over the routing table.

Future Work

• fix performance issues, in particular:
  • The GNUnet protocol has no way to reply that content is not available. Thus the current implementation of the clients just keeps trying -- forever (unless the user aborts). What would be better is using the size of GNUnet (estimated via number of hosts known) to determine by software when to better give up.
• improve documentation
• translate (documentation, user interface, messages, man-pages), gettext support in the code and build-system
• more powerful user-interfaces (preferences, non-gtk+).
• write more extractors for libextractor (contact)
• How about pseudonym based E-mail with accounting to fight Spam? E-mail without a central server at some ISP would also get rid of the pain with changing the address ever again.
• port to win32 (contact)
• fix bugs
• port to OS X (contact)
• add directories and namespaces (contact)
• implement other transport protocols (other than UDP and TCP). We have SMTP contact), planned are HTTP and IPv6 (contact).
  Feel free to add smoke signals and others to this list as you please.

• distributed computation (sandboxing!?)
• add steganography to existing transport protocols

Footnotes

NBO - network byte order

Copyright (C) 2002, 2003 Christian Grothoff.

Verbatim copying and distribution of this entire article 
is permitted in any medium, provided this notice is preserved.