In the box where all the cables are run, it's only wired for phone. The cable can carry Ethernet, but it has to be terminated correctly.
The jacks are RJ45 so you'd only need a new Ethernet distribution block. Make sure both ends are terminated the same - 568A or 568B - you can't mix them.
Do you still have phones? If so, you'll need to keep the lines for the phones terminated in the current block and move the computer lines to an Ethernet block.