Abstract away matrices

Change coder API to not use matrices on the outside.

This is confusing as it is not really clear what is what and what goes where.