Really interesting article. However isn't it a serious problem that the ethical concerns you describe as baked in already move beyond what you're and others are thinking? Once it becomes more advanced and thinks more like people with the more information you give it could it not simply decide certain ethical principles are impractical and/or unwise? Could it not simply begin to favour something like utility based on an algorithm at the expense of other types of goods?
I always thinks ethics are difficult precisely because as people we tend to think about the trade offs and the background. Very rarely is an ethical situation cut and dry in the way that your experiment with chat gpt kind of indicated. This in some way makes it unsatisfactory and perhaps indicates that a mere move towards liberal rights is 'naturally good'. However, liberal norms and rights are culturally contingent even if we may want to envision them as universal goods.
Really interesting article. However isn't it a serious problem that the ethical concerns you describe as baked in already move beyond what you're and others are thinking? Once it becomes more advanced and thinks more like people with the more information you give it could it not simply decide certain ethical principles are impractical and/or unwise? Could it not simply begin to favour something like utility based on an algorithm at the expense of other types of goods?
I always thinks ethics are difficult precisely because as people we tend to think about the trade offs and the background. Very rarely is an ethical situation cut and dry in the way that your experiment with chat gpt kind of indicated. This in some way makes it unsatisfactory and perhaps indicates that a mere move towards liberal rights is 'naturally good'. However, liberal norms and rights are culturally contingent even if we may want to envision them as universal goods.