utf 8 - How to decompress a []byte content in gzip format that gives an error when unmarshaling -

i'm making request api, []byte out of response (ioutil.readall(resp.body)). i'm trying unmarshal content, seems not encoded on utf-8 format, unmarshal returns error. i'm trying so:

package main  import (     "encoding/json"     "fmt"      "some/api" )  func main() {     content := api.someapi.somerequest() // []byte variable     var data interface{}     err := json.unmarshal(content, &data)     if err != nil {         panic(err.error())     }     fmt.println("data response", data) } 

i error invalid character '\x1f' looking beginning of value. record, response includes in header content-type:[application/json; charset=utf-8].

how can decode content avoid these invalid characters when unmarshaling?

edit

this hexdump of content: play.golang.org/p/oj5mqeramj

judging hex dump receiving gzip encoded data you'll need use compress/gzip decode first.

try this

package main  import (     "bytes"     "compress/gzip"     "encoding/json"     "fmt"     "io"     "some/api" )  func main() {     content := api.someapi.somerequest() // []byte variable      // decompress content io.reader     buf := bytes.newbuffer(content)     reader, err := gzip.newreader(buf)     if err != nil {         panic(err)     }      // use stream interface decode json io.reader     var data interface{}     dec := json.newdecoder(reader)     err = dec.decode(&data)     if err != nil && err != io.eof {         panic(err)     }     fmt.println("data response", data) } 

previous

character \x1f unit separator character in ascii , utf-8. never part of utf-8 encoding, can used mark off different bits of text. string \x1f can valid utf-8 not valid json far know.

i think need read api specification closely find out using \x1f markers for, in meantime try removing them , see happens, eg

import (     "bytes"     "fmt" )  func main() {     b := []byte("hello\x1fgoodbye")     fmt.printf("b %q\n", b)     b = bytes.replace(b, []byte{0x1f}, []byte{' '}, -1)     fmt.printf("b %q\n", b) } 

prints

b "hello\x1fgoodbye" b "hello goodbye" 

playground link