First, the style of the bust found in the river was realism, a style of art attempting to represent subject matter truthfully, which was a prevalent art style of the time. This lines up with the time period that the statue is believed to have been created in.
Second, the bust resembles other works of Julius Caesar. The hair line is very similar to that of the
image on coins depicting him. Because of the resemblance between known images of Julius Caesar it is probably his 50-year-old self-statue.
And finally, people of this time period often buried this bust in order to show reverence for Julius Caesar. This was due to the unfortunate circumstances of his demise and how many people had respected him as a political official.