I did some Google search about yellow discharge on male dogs and found mostly comments about dog's smegma which is apparently a normal thing on male dogs.
We are not convinced, however, so this morning I took Oliver to the vet along with some urine sample. We spent almost 1 hour at the vet office waiting to get his urinalysis done.
Good thing we didn't just believe what we found on the internet. The vet said there is a lot of bacteria and white blood cells in Oliver's urine, so he definitely got UTI. His urine pH is also high and they found some crystals in his urine as well. The vet gave him antibiotics and anti-inflammatory to help with the pain while waiting for the antibiotics to kick in. She also asked us to come back after 10 days to check on his urine again, especially the pH. She said that the pH should go down, but if not, then we may need to give Oliver some acidifier or change his diet to reduce the pH. Apparently the high pH is causing the crystals to form and those crystals can turn into stones which in turn will cause kidney stone problem.
In any case, we are glad that we now know what's going on with Oliver. We are also glad that we caught this infection quite early before it spreads and becomes a full-blown bladder infection. Hope he feels better soon!