You must start a company, if you are conducting business in Japan (i.e. buying and selling products or services). If you are not conducting business but, for example, have an office doing advertising, public relations, market research, etc., you are not required to establish a company.
Aside from the legal requirements, if your Japan office has at least one employee, you most likely will need to provide social insurance (health, pension, unemployment, and workers’ accident compensation insurances). Providing that insurance requires doing a Japanese payroll, and that requires tax withholding and insurance payments. Such payments must be made from a domestic Japanese bank account, and bank accounts must be owned by either an individual who is resident in Japan or by a Japanese company. Most companies find having their bank account owned by an employee unacceptable from a control standpoint, and, therefore, form a legal entity in order to have a domestic bank account. Another alternative is to use HTM’s banking service using the HTM legal entity. This approach can be useful to small operations and provides a high degree of control, but has the disadvantage that the name of the account must include the HTM name.
Another reason for having a corporate bank account (and therefore needing a company) is that many customers in Japan will only pay to a domestic Japanese bank account. While there are companies that sell directly to customers in Japan from abroad, they often find that customers insist on paying to a domestic Japanese bank account, and therefore end up needing a Japanese bank account, and that requires having a company in Japan.