FTP is short for File Transfer Protocol.
Your website is hosted on a machine somewhere on the internet (local or in another country).
Within your hosting setup is your account that contains all your website files and data. This is a very similar setup to your own computer. So think of your account as your desktop and it contains Folders (directories) and documents.
FTP allows you to log into the your hosting account and upload, download and edit the files in a similar way that you manage files on your desktop.
The software that allows us to use FTP is called a “FTP client”
To access your hosting via FTP your will need the server address, username and password (sometimes you may need a custom post number and additional security). Your host will provide you with these details.
There are many FTP clients available. Here are a few
Some more information can be found here http://www.bleepingcomputer.com/tutorials/introduction-to-ftp/