You can group by the expression:
DATEADD(hour, DATEDIFF(hour, 0, XDateTime), 0)
e.g.
SELECT
DATEADD(hour, DATEDIFF(hour, 0, XDateTime), 0) AS XHour,
AVG(XValue) AS XAvg
FROM MyTable
GROUP BY DATEADD(hour, DATEDIFF(hour, 0, XDateTime), 0)
ORDER BY XHour