Solution 1 (Accepted):

You already have all (median, Q1, and Q3), and just need to assign lower, upper, middle, ymin, and ymax

(FYI, https://en.wikipedia.org/wiki/Box_plot#/media/File:Boxplot_vs_PDF.svg)

ggplot(depth, aes(Sample, median)) +
geom_boxplot(aes(lower = granular_first_quartile, upper = granular_third_quartile, 
middle = median, 
ymin = granular_first_quartile - 1.5*(granular_third_quartile-granular_first_quartile), 
ymax = granular_third_quartile+1.5*(granular_third_quartile-granular_first_quartile)),
stat="identity")+ coord_flip()