In the greater historical scene, the sources of conflict between Japan and Russia have been comparatively recent. The core of Japanese interests have always, understandably, been concentrated in East Asia, and Russia didn't become a presence in this area until the 1600's. It was at this point that the sources of conflict between the two countries could come to the fore.
The initial hot spot was Hokkaido, then known as Ezo by the Japanese. Up until the second half of the 1800's, most of Hokkaido was nominally under the control of the Shogunate but was, in fact, mostly populated by the indigenous Ainu. After having already expanded east to the Pacific, the Russians took an interest in the islands off their Pacific coast such as Sakhalin, the Kurils, and Hokkaido. At this moment, the Japanese were still in the Edo Period (~1600-1860's), so there hadn't been as much of a focus on foreign expansion for some time. However, a Russian presence in these islands, especially Hokkaido, could have been an easy springboard to jump to the rest of the Japanese archipelago. This prompted first greater Shogunate control in the early 1800's and then mass Japanese settlement in the island later in the century.
When the Edo Period came to an end, Japan decided that having their own imperial sphere of influence in East Asia was essential for their security as a block against the European empires. In the last few decades of the 1800's and the first years of the 1900's, Japan expanded their empire to include Korea, pieces of the Chinese coast, Taiwan, etc.
On the flip side, the Russians were (and still are, to an extent) always interested in finding warm water ports that would enable them to more effectively project sea power and have access to overseas trade. In East Asia, that would mean controlling usable ports in Chinese territory such as the Liaodong Peninsula, to the west of modern-day North Korea. This was, however, a piece of territory that both the Russians and Japanese wished to control. It was the struggle for this territory as well as a controlling presence in the region that the Japanese and Russians fought the aptly named Russo-Japanese War in 1904-5. In this effort, the Japanese had the support of the British, who had been suspicious of Russian intentions in Asia for some time due to their desire to protect their own Asia colonies such as the British Raj in India. The ultimate Japanese victory in this war led to territorial gains in Sakhalin and assurances for their sphere of influence, but the American-mediated treaty left a number of issues unresolved.
The Bolshevik Revolution in Russia in 1917 didn't fundamentally change the struggle over a greater sphere of influence in East Asia, but it temporarily brought about a power vacuum in Russia during the civil war between the so-called Reds and Whites. The Japanese were part of the larger coalition of countries to send forces against the Bolsheviks in the 1920's, but the Soviet Union settled into existence regardless. Following this, there was a short gap in direct hostilities between the Soviet Union and Japan, though the former offered ample support for both the Chinese Nationalists and Communists since they wished to find some political partners in Asia. Especially when the Japanese began a new policy of imperial expansion into China in the 1930's, this led to an obvious source of tension. The Japanese and Soviets came to blows in the 1930's in modern-day Manchuria and Mongolia. The battles themselves ultimately wound up as a Soviet victory, but the border between the two countries remained unchanged for now.
Going into the 1940's, the Soviets directed far more of their attention to the growing threat from Nazi Germany. Though the Japanese became allies with the Germans, they were already busy enough in China and would become even more so after striking the United States, Britain, and the Netherlands in 1941, so they didn't have the resources to handle those conflicts and add a new one with the Soviet Union. Since the Soviets would be busy with the Germans until May of 1945, there wouldn't be a push from the other side for the time being either. This changed once Germany had been compelled to surrender, and Joseph Stalin had been persuaded to join the Sino-Anglo-American led effort against the Japanese in August 1945, just before the end of the war. The Soviets overwhelmed Japanese positions in Manchuria and Korea and seized the entirety of Sakhalin as well as the Kuril Islands, just to the northeast of Hokkaido. In fact, Stalin had proposed to US President Harry Truman to divide Hokkaido between Soviet and American spheres of influence, but the latter rejected this.
Once the war was over, the Americans led the occupation of Japan, and the political and economic keys to power in Japan favored closer ties to America as a shield against potential future Soviet expansion as well as Soviet influence among left-wing groups in Japan. As such, Japan would sign a treaty of "mutual cooperation and security" with the United States and remain in the pro-Capitalist camp throughout the Cold War. Technically speaking, Japan and the Soviet Union/Russia never signed a treaty resolving the issues of World War II. Apart from Japan's close relations to the United States, the true main reason for this is the unresolved status of the Kuril Islands just offshore from Hokkaido. Japan still claims ownership of the islands, though they have been under Russian control since the end of the war. Joseph Stalin secretly obtained permission from the other wartime allies to take these islands as well as Sakhalin, which also remains under Russian control.
As things stand now, Japan and Russia are neither political/military friends nor enemies, despite the lack of a peace treaty. The issue of the Kuril Islands as well as fishing rights surrounding Russian and Japanese territory continues to be a hot spot, but neither country has indicated any interest in militarily forcing the matter. Naturally, the ongoing close relations between Japan and the United States means that Russo-Japanese relations won't be especially close so long as things remain as they are. In addition, Russia has ongoing ties to North Korea that Japan is aware of. This is both a potential source of conflict, for obvious reasons, but also a potential opportunity as contacts with the Russians could lead to a larger regional approach to North Korean issues. For the time being at least, they both still share a presence in the same region (East Asia) and therefore naturally have overlapping interests. Some of those are mutually exclusive; some aren't.
I hope that this helps!