You could use all of them if you wanted to but best way to use them is like this
JPG - solid images with a lot of detailor part of the content. Background images.
PNG - if you have transparent backgrounds such as logos or icons.
SVG - Again if you have a logo or icons you want to scale you can use this format
I don't really use GIF at all as PNG replaces its use because of quality.
Of course there is a image format called WebP that is used to help optimize images to make them load faster as well.