In engineering and economics a certain vector of inputs or decisions must often be chosen, subject to some constraints, such that the expected costs arising from the deviation between the output of a stochastic linear system and a desired stochastic target vector are minimal. In many cases the loss function u is convex and the occuring random variables have, at least approximately, a joint discrete distribution. Concrete problems of this type are stochastic linear programs with recourse, portfolio optimization problems, error minimization and optimal design problems. In solving stochastic optimization problems of this type by standard optimization software, the main difficulty is that the objective function F and its derivatives are defined by multiple integrals. Hence, one wants to omit, as much as possible, the time-consuming computation of derivatives of F. Using the special structure of the problem, the mathematical foundations and several concrete methods for the computation of feasible descent directions, in a certain part of the feasible domain, are presented first, without any derivatives of the objective function F. It can also be used to support other methods for solving discretely distributed stochastic programs, especially large scale linear programming and stochastic approximation methods.