This is the README.txt file for the TME distribution.  The TME package allows
a program running on a Linux workstation to easily transfer data to and from
a user's circuit in an Terasic/Altera DE-4 or DE-5 demonstration board
over the PCIe bus.


DOCUMENTATION

There are 4 pdf documents in the doc sub-directory, entitled:

	The TME Ports Package
	Creating and Running Circuits with the TME Ports Package
	An Example Application for the TME Package
	The tmemon Program


INSTALLATION

1) Untar the package and change into the top level directory:

	tar xzf tme1.3.tar.gz
	cd tme1.3

2) Edit the top level makefile and change the top two lines from this to
something sensible:

	INSTALL_DIR=please_set_the_INSTALL_DIR_variable_in_the_top_level_makefile_and_run_make_config
	DEFAULT_SERVER=localhost

Set INSTALL_DIR to a directory where the package's files should be installed.
Set DEFAULT_SERVER to the name of the machine that is connected to the
demonstration board through the PCIe bus.

3) Type:

	make config
	make install

This will copy the values of the INSTALL_DIR and DEFAULT_SERVER variables
to the lower level makefiles that need them, and then compile and install the 
programs and scripts that make up the package.


KERNEL DRIVER

To communicate with a circuit in the board over the PCIe bus, a device
driver must be installed into the Linux kernel.  The source for the DE-4 driver
is in src/de4_kernel_driver, and the DE-5 driver is in src/de5_kernel_driver.
Each directory contains a compiled driver, but you will likely have to
re-compile the source to get a binary that will work with your kernel.
To re-compile the source, cd to that directory and type
"make".  You should see something like this:

make -C /lib/modules/3.2.0-30-generic/build M=/home/drg/tme1.3/src/de4_kernel_driver modules
make[1]: Entering directory `/usr/src/linux-headers-3.2.0-30-generic'
  CC [M]  /home/drg/tme1.3/src/de4_kernel_driver/pcieports.o
/home/drg/tme1.3/src/de4_kernel_driver/pcieports.c: In function ‘write_chunk’:
/home/drg/tme1.3/src/de4_kernel_driver/pcieports.c:429:15: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
/home/drg/tme1.3/src/de4_kernel_driver/pcieports.c: In function ‘read_chunk’:
/home/drg/tme1.3/src/de4_kernel_driver/pcieports.c:478:15: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/drg/tme1.3/src/de4_kernel_driver/pcieports.mod.o
  LD [M]  /home/drg/tme1.3/src/de4_kernel_driver/pcieports.ko
make[1]: Leaving directory `/usr/src/linux-headers-3.2.0-30-generic'

To put the driver into a directory where your kernel can find it, type
"make modules_install" (as root).  You should see something like this:

make -C /lib/modules/3.2.0-30-generic/build M=/home/drg/tme1.3/src/de4_kernel_driver modules_install
make[1]: Entering directory `/usr/src/linux-headers-3.2.0-30-generic'
  INSTALL /home/drg/tme1.3/src/de4_kernel_driver/pcieports.ko
  DEPMOD  3.2.0-30-generic
make[1]: Leaving directory `/usr/src/linux-headers-3.2.0-30-generic'
depmod -a

Each user circuit that uses the tme package to communicate over the PCIe
bus contains a PCIe IP core.  Before the driver can talk to the circuit,
the Linux kernel and the PCIe IP core must agree on addresses and other
configuration information.

Unfortunately this means that every time the FPGA is reloaded, the PCIe IP
core must re-negotiate this information with the kernel.  This complicates
the process of programming a new circuit into the FPGA.

The first time the FPGA is programmed after a reboot, you must:

	1) program the FPGA using Quartus
	2) reboot the Linux machine, without powering it off and on

After that first reboot, the Linux kernel will remember that the board is
present on the PCIe bus, and you should not have to reboot again, even if
entirely new circuits are programmed into the board, as all of the circuits
will look the same to the kernel.

However, the kernel will still have to re-negotiate with the PCIe IP core
every time the FPGA is re-programmed.  To re-program the FPGA, other than the
first time, you must:

	1) program the FPGA using Quartus
	2) (as root) echo 1 > /sys/bus/pci/devices/0000:03:00.0/remove
	3) (as root) echo 1 > /sys/bus/pci/rescan

The "tmrun" command script can be used to do step 1).  However, steps 2) and 3)
must be run as root, and are not part of tmrun.

Note that in step 2), the "0000:03:00.0" part of the path is an example, and
will likely be different on your machine.  You can find the correct string by
looking for the board in the output of the lspci command after the first
reboot.  Look for a line like this, which is a DE-4:

# lspci | grep Altera
03:00.0 Unassigned class [ff00]: Altera Corporation Device 1337 (rev 01)

(A DE-5 is similar, but says it is a "Device 1338", instead of 1337).

You should see lines printed by the kernel when the driver runs that look
like this:

# dmesg
...
[104615.130389] pci 0000:03:00.0: reg 10: [mem 0xf7f00000-0xf7f07fff]
[104615.150451] pci 0000:03:00.0: BAR 0: assigned [mem 0xf7f00000-0xf7f07fff]
[104615.150454] pci 0000:03:00.0: BAR 0: set to [mem 0xf7f00000-0xf7f07fff] (PCI address [0xf7f00000-0xf7f07fff]
[104615.150489] pcieports 0000:03:00.0: registering driver
[104615.150501] pcieports 0000:03:00.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24
[104615.150725] pcieports 0000:03:00.0: physical 00000000f7f00000 mapped to ffffc90011970000
[104615.150743] pcieports 0000:03:00.0: irq 72 for MSI/MSI-X
[104615.150755] pcieports 0000:03:00.0: allocated device node major 250 minor 0
[104615.150786] pcieports 0000:03:00.0: setting latency timer to 64
...


ENTRY in /dev

Linux user programs communicate with the driver through an entry in the
/dev directory.  The DE-4 kernel driver creates an entry named /dev/pcieports,
but by default it is only usable by root.  The tme software expects it
to be called /dev/tme_portmux0, and be usable by all users.  To fix both of
these problems, copy the file src/udev_rules/53-pcieports.rules to the
directory /etc/udev/rules.d on your machine (as root).  This file will cause
the udev daemon on your machine to fix the name and the permissions whenever
the driver is re-loaded.

The DE-5 kernel driver creates a /dev entry named /dev/de5_pcieports, and the
53-pcieports.rules file will fix it as well, calling it /dev/tme_de5_portmux0.


TMEMON

Once the board is programmed and the /dev entries have been created, you
should be able to start the tmemon daemon program on the Linux machine that
contains the DE-4 or DE-5 board.

In any directory on that machine, run:

	tmemon &

If the board has been programmed with a user's circuit that includes the
TME portmux circuitry and the driver has been re-loaded, tmemon should print:

	Opening interface...
	Successfully opened interface.

Once tmemon is running, it should stay running indefinitely until killed.
It will accept requests from user programs using the TME package, and will
relay them to the board over the PCIe bus.


EXAMPLE CIRCUITS

There are two example circuits in the examples/de4 directory, along with
programs that use the TME package to communicate with them.  To compile
one of the programs, cd to the directory and type "make".  To compile one of
the circuits using Quartus, type "make bits", or use the Quartus GUI directly.
Read the document entitled "An Example Application for the TME Package" for
complete details on how to build and use one of the examples.

The "counter" circuit implements a simple 32-bit counter, which can be
read by the program.  The "summer" circuit reads 32-bit values from
the program, and will return their sum.

The same two circuits can be found in the examples/de5 directory, and can
be used with the DE-5.


OTHER NOTES

The tme script that builds the pcie_portmux component for your circuit assumes
that Altera's quartus bin directory is on your search path.

The optional tmstatus program needs the tcl/tk wish program installed.  We are
using tk 8.5.

The optional tmget, tmrelease and tmrun scripts assume that the TME program "tm"
is on your search path.

By default, the ports package will assume that it is talking to a board that
is connected to "localhost".  You can over-ride the default host by setting the
TM_SERVER environment variable before opening the port.  To change this
default permanently, modify the DEFAULT_SERVER variable in src/tmports/makefile.


JAVA SUPPORT

There is preliminary support that should allow java programs to talk to
circuits in src/tmports/*.java.  It is still a bit rough, but we have used it
on an Android phone to talk to an FPGA circuit over a wireless LAN.
