Ttdfjt

Consider the equality
beginequationexpleft(-fracihbarboldsymbolphi Jright)left|xright>=left|R(phi)xright>,endequation
where $left|xright>$ denotes a position eigenstate, $J$ the total angular momentum operator on the ket space, and $R(boldsymbolphi)$ the $mathbbR^3$ rotation matrix for rotations around $boldsymbolphi=phicdotmathbfn$ with $||mathbfn||=1$.

Some consider this equation to be the definition of total angular momentum. How can this equation be proven by using the arguably more popular and classically motivated definition?
beginequation
mathbfJ:=mathbfL+mathbfS=mathbfXtimesmathbfP+mathbfS
endequation

Note: Thanks to the work of @Valter Moretti and @Adam Latosiński an equality between the two defintions has been established for a spinless particle.

Consider a smooth function $psi$ in $L^2(mathbbR^3, d^3x)$ (more precisely $psi$ is assumed to belong to Schwartz' space) and define for a fixed $n in mathbbS^2$ and $phi in mathbbR$
$$psi_phi(x)= psi(R^-1_n(phi)x)$$
Since $R^-1_n(phi) = e^-phi n cdot S$, where $S= (S_x,S_y,S_z)$ are the three generators of $SO(3)$:
$$(S_j)_rs= epsilon_jrs$$
we have
$$fracddphipsi_phi(x) = -sum_j,r,s=1^3 n_j epsilon_jrsx_rfracpartial partial x_spsi_phi(x):.tag1$$
Now consider
$$psi'_phi(x)= left(e^fracihbarphi n cdot Jpsiright)(x):,$$
where, omitting the spin part of $J$ since as far as I unserstand, you are interested in the spatial part of the state only,
$$J_j = sum_r,s=1^3epsilon_jrs X_r P_s = -ihbar sum_r,s=1^3epsilon_jrs X_r fracpartialpartial x_s $$
Computing the $phi$-derivative (using Stone's theorem and some careful analysis about the use of different topologies. I do not want to enter into the details here, I just say that here smoothness of $psi$ matters), we have
$$fracddphipsi'_phi(x) = fracihbarsum_j=1^3 n cdot left(J_jpsi'_phiright)(x)= -sum_j,r,s=1^3 n_j epsilon_jrsx_rfracpartial partial x_spsi'_phi(x):.tag2$$
In summary, for $x$ fixed, $psi_phi(x)$ and $psi'_phi(x)$ satisfy the same first order differential equation (in normal form with smooth known term) and furthermore they satisfy the same initial condition
$$psi_0(x)=psi'_0 (x):.$$
The uniqueness theorem for the solutions of first-order differential equations implies that
$$psi_phi(x)=psi'_phi (x):.$$
In other words
$$left(e^-fracihbar phi n cdot Jpsiright)(x) = psileft(R_n^-1(phi)xright):.$$
(The result extends to the whole $L^2$ space made als of non-smooth functions exploiting the fact that Schwartz' space is dense therein.)
The final step is quite formal, but it can be made rigorous adopting the theory of rigged Hilbert spaces to make rigorous the bra-ket notation for improper eigenvectors of the position operators. In position representation $|yrangle = delta(x-y)= delta_y(x)$, so that, using rotational invariance of the Dirac delta function,
$$left(e^-fracihbar phi n cdot Jdelta_yright)(x) = delta_yleft(R_n^-1(phi)xright)= deltaleft(R_n^-1(phi)x -yright)= deltaleft(x -R_n(phi)yright):.$$
Coming back to the abstract notation, the found identity reads
$$e^-fracihbar phi n cdot J|yrangle = |R_n(phi)yrangle:.$$

(A completely rigorous proof can be found in this book of mine.)

The most broad definition of the angular momentum operator $bf J$ is the operator that generates rotation, so the equality you want to prove is just the definiton (assumin that the particle we consider does not have any internal degrees of freedom, which could include spin) and as such does not require a proof.

But what would then needs a proof is the relation between the angular momentum operator and the position operator $$ bf J = bf X times bf P$$
Let's see that this is satisfied for $bf J_3$, i.e. $$ bf J_3 = bf X_1bf P_2 - bf X_2bf P_1$$
(other components will be analogous).

Let us consider an rotation acting on a state $|psirangle = int psi(x) |xrangle dx$.
$$ exp(- fracihbarphi, bf J_3) int psi(x)|xrangle dx = int psi(x)|R(phi)xrangle dx = int psi(R^-1(phi)x) |xrangle$$
That means that
$$ bf J_3 |psirangle = ihbar left.fracddphiright|_phi=0 exp(- fracihbarphi, bf J_3) int psi(x)|xrangle dx = ihbar int left.fracddphiright|_phi=0 psi(R^-1(phi)x) |xrangle$$

We have $$psi(R^-1(phi)x) = psi(x_1 cosphi + x_2 sinphi,-x_1sinphi+x_2cosphi,x_3) $$
so
$$ left.fracddphiright|_phi=0 psi(R^-1(phi)x) = x_2 fracpartial psi(x)partial x_1 - x_1 fracpartial psi(x)partial x_2$$
so
beginalign bf J_3|psirangle &= ihbar int Big(x_2 fracpartial psi(x)partial x_1 - x_1 fracpartial psi(x)partial x_2Big) |xrangle = \
&= -bf X_2 int Big(-ihbarfracpartial psi(x)partial x_1Big) |xrangle dx + bf X_1 int Big(-ihbarfracpartial psi(x)partial x_2Big) |xrangle = \ &= -bf X_2 bf P_1 int psi(x) |xrangle dx + bf X_1 bf P_2 int psi(x) |xrangle = \ &= (bf X_1bf P_2 - bf X_2bf P_1)|psirangle endalign
so $$ bf J_3 = bf X_1bf P_2 - bf X_2bf P_1$$

If a particle has internal degrees of freedom, then $$ bf J = bf X times bf P + bf S$$
where $bf S$ is the spin operator acting on these internal degrees of freedom.

Let's just consider $e^iJ_zphi$ where $hbar=1$.
We have
$$R_z=beginbmatrixcos(phi) & sin(phi)&0\-sin(phi)&cos(theta) & 0 \0&0&1endbmatrix$$
$$J_z=frac1ifracdR_z(phi)dphi|_phi=0=beginbmatrix0 & -i&0\i& 0 & 0 \0&0&0endbmatrix$$

We can exapnd exponential as:
$$e^iJ_zphi=1+iJ_zphi-J_z^2phi^2/2!-iJ_z^3phi^3/3!+....$$
Or
$$e^iJ_zphi=beginbmatrix1 & 0&0\0 & 1 & 0 \0&0&1endbmatrix+
phi beginbmatrix0 & 1&0\-1 & 0 & 0 \0&0&0endbmatrix
+phi^2/2! beginbmatrix-1 & 0&0\0& -1 & 0 \0&0&0endbmatrix+ ...=R_z(phi)$$
We can do the same for other axes.
Check out Quantum Field Theory by Lewis H. Ryder/
Single-particle relativistic wave equation.